2015-11-09 13:25:07 [DEBUG   ] Got command RPT from MCS: ref #528033
2015-11-09 13:25:07 [DEBUG   ] summary = NORMAL
2015-11-09 13:25:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528033  14 57335 48307351 A NORMALNORMAL'
2015-11-09 13:25:09 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-09 13:25:29 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 13:25:49 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 13:25:50 [DEBUG   ] Finished updating board flags in 0.994 seconds
2015-11-09 13:26:09 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:26:29 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:26:49 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 13:26:51 [DEBUG   ] Finished updating board flags in 1.808 seconds
2015-11-09 13:27:09 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 13:27:29 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-09 13:27:48 [DEBUG   ] Got command RPT from MCS: ref #528042
2015-11-09 13:27:48 [DEBUG   ] summary = NORMAL
2015-11-09 13:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528042  14 57335 48468539 A NORMALNORMAL'
2015-11-09 13:27:48 [DEBUG   ] Got command RPT from MCS: ref #528043
2015-11-09 13:27:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 13:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528043  48 57335 48468539 A NORMALSystem calibrated and operating normally'
2015-11-09 13:27:49 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 13:27:50 [DEBUG   ] Finished updating board flags in 1.234 seconds
2015-11-09 13:28:09 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 13:28:29 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 13:28:49 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 13:28:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 13:28:50 [DEBUG   ] Finished updating board flags in 1.227 seconds
2015-11-09 13:29:09 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 13:29:29 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 13:29:49 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 13:29:50 [DEBUG   ] Finished updating board flags in 1.180 seconds
2015-11-09 13:30:07 [DEBUG   ] Got command RPT from MCS: ref #528062
2015-11-09 13:30:07 [DEBUG   ] summary = NORMAL
2015-11-09 13:30:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528062  14 57335 48607435 A NORMALNORMAL'
2015-11-09 13:30:09 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 13:30:29 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2015-11-09 13:30:49 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 13:30:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 13:30:50 [DEBUG   ] Finished updating board flags in 1.028 seconds
2015-11-09 13:31:09 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 13:31:29 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 13:31:49 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 13:31:51 [DEBUG   ] Finished updating board flags in 1.609 seconds
2015-11-09 13:32:09 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2015-11-09 13:32:29 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-09 13:32:48 [DEBUG   ] Got command RPT from MCS: ref #528066
2015-11-09 13:32:48 [DEBUG   ] summary = NORMAL
2015-11-09 13:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528066  14 57335 48768547 A NORMALNORMAL'
2015-11-09 13:32:48 [DEBUG   ] Got command RPT from MCS: ref #528067
2015-11-09 13:32:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 13:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528067  48 57335 48768547 A NORMALSystem calibrated and operating normally'
2015-11-09 13:32:49 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-09 13:32:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-09 13:32:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 13:32:51 [DEBUG   ] Finished updating board flags in 1.530 seconds
2015-11-09 13:33:09 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-09 13:33:29 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-09 13:33:49 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 13:33:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 13:33:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 13:33:51 [DEBUG   ] Finished updating board flags in 1.409 seconds
2015-11-09 13:34:09 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-09 13:34:29 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-09 13:34:49 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-09 13:34:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 13:34:51 [DEBUG   ] Finished updating board flags in 1.383 seconds
2015-11-09 13:35:07 [DEBUG   ] Got command RPT from MCS: ref #528084
2015-11-09 13:35:07 [DEBUG   ] summary = NORMAL
2015-11-09 13:35:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528084  14 57335 48907511 A NORMALNORMAL'
2015-11-09 13:35:09 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-09 13:35:29 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-09 13:35:49 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-09 13:35:51 [DEBUG   ] Finished updating board flags in 1.215 seconds
2015-11-09 13:36:09 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-09 13:36:29 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-09 13:36:49 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-09 13:36:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 13:36:51 [DEBUG   ] Finished updating board flags in 1.241 seconds
2015-11-09 13:37:09 [DEBUG   ] Finished updating ant stats in 0.397 seconds
2015-11-09 13:37:30 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 13:37:48 [DEBUG   ] Got command RPT from MCS: ref #528093
2015-11-09 13:37:48 [DEBUG   ] summary = NORMAL
2015-11-09 13:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528093  14 57335 49068551 A NORMALNORMAL'
2015-11-09 13:37:48 [DEBUG   ] Got command RPT from MCS: ref #528094
2015-11-09 13:37:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 13:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528094  48 57335 49068551 A NORMALSystem calibrated and operating normally'
2015-11-09 13:37:50 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 13:37:51 [DEBUG   ] Finished updating board flags in 1.167 seconds
2015-11-09 13:38:10 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 13:38:30 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 13:38:50 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 13:38:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 13:38:51 [DEBUG   ] Finished updating board flags in 1.057 seconds
2015-11-09 13:39:10 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2015-11-09 13:39:30 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:39:50 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 13:39:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 13:39:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 13:39:52 [DEBUG   ] Finished updating board flags in 1.992 seconds
2015-11-09 13:40:07 [DEBUG   ] Got command RPT from MCS: ref #528113
2015-11-09 13:40:07 [DEBUG   ] summary = NORMAL
2015-11-09 13:40:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528113  14 57335 49207579 A NORMALNORMAL'
2015-11-09 13:40:10 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 13:40:30 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:40:50 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:40:51 [DEBUG   ] Finished updating board flags in 1.190 seconds
2015-11-09 13:41:10 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:41:30 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:41:50 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999960
2015-11-09 13:41:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999960
2015-11-09 13:41:51 [DEBUG   ] Finished updating board flags in 1.049 seconds
2015-11-09 13:42:00 [DEBUG   ] Got command TBN from MCS: ref #528117
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '1', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '2', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '3', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] tbn = exited with status True, code 0x00
2015-11-09 13:42:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBN   528117   8 57335 49320752 A NORMAL'
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '4', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '5', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '6', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '7', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '8', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '9', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '10', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '11', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '12', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '13', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '14', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '15', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '16', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '17', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '18', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '19', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '20', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '21', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '22', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '23', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '24', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '25', '1447076522']
2015-11-09 13:42:00 [DEBUG   ] spawn_process: at 2015-11-09 13:42:00 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '26', '1447076522']
2015-11-09 13:42:00 [INFO    ] Finished spawning TBN in 0.045 s
2015-11-09 13:42:10 [DEBUG   ] Finished updating ant stats in 1.068 seconds
2015-11-09 13:42:30 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 13:42:48 [DEBUG   ] Got command RPT from MCS: ref #528118
2015-11-09 13:42:48 [DEBUG   ] summary = NORMAL
2015-11-09 13:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528118  14 57335 49368547 A NORMALNORMAL'
2015-11-09 13:42:48 [DEBUG   ] Got command RPT from MCS: ref #528119
2015-11-09 13:42:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 13:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528119  48 57335 49368547 A NORMALSystem calibrated and operating normally'
2015-11-09 13:42:50 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 13:42:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 13:42:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 13:42:52 [DEBUG   ] Finished updating board flags in 1.755 seconds
2015-11-09 13:43:10 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 13:43:30 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:43:50 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:43:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-09 13:43:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-09 13:43:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-09 13:43:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-09 13:43:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-09 13:43:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-09 13:43:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-09 13:43:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-09 13:43:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-09 13:43:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-09 13:43:52 [DEBUG   ] Finished updating board flags in 1.623 seconds
2015-11-09 13:44:10 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 13:44:30 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 13:44:50 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999960
2015-11-09 13:44:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999960
2015-11-09 13:44:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999960
2015-11-09 13:44:52 [DEBUG   ] Finished updating board flags in 1.565 seconds
2015-11-09 13:45:07 [DEBUG   ] Got command RPT from MCS: ref #528139
2015-11-09 13:45:07 [DEBUG   ] summary = NORMAL
2015-11-09 13:45:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528139  14 57335 49507663 A NORMALNORMAL'
2015-11-09 13:45:10 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:45:30 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 13:45:50 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 13:45:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 13:45:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 13:45:52 [DEBUG   ] Finished updating board flags in 1.432 seconds
2015-11-09 13:46:10 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:46:11 [DEBUG   ] Got command TBW from MCS: ref #528146
2015-11-09 13:46:11 [DEBUG   ] tbw = exited with status True, code 0x00
2015-11-09 13:46:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBW   528146   8 57335 49571847 A NORMAL'
2015-11-09 13:46:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 pkill tbn
2015-11-09 13:46:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 pkill tbn
2015-11-09 13:46:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 pkill tbn
2015-11-09 13:46:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 pkill tbn
2015-11-09 13:46:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 pkill tbn
2015-11-09 13:46:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 pkill tbn
2015-11-09 13:46:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 pkill tbn
2015-11-09 13:46:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 pkill tbn
2015-11-09 13:46:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 pkill tbn
2015-11-09 13:46:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 pkill tbn
2015-11-09 13:46:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 pkill tbn
2015-11-09 13:46:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 pkill tbn
2015-11-09 13:46:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 pkill tbn
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa27', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '1', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa24', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '2', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa23', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '3', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa18', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '4', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa10', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '5', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa32', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '6', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa11', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '7', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa30', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '8', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa12', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '9', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa8', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '10', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa13', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '11', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa26', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '12', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa14', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '13', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa22', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '14', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa20', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '15', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa1', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '16', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa5', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '17', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa25', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '18', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa35', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '19', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa19', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '20', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa21', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '21', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa6', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '22', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa7', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '23', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa33', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '24', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa15', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '25', '12000000']
2015-11-09 13:46:15 [DEBUG   ] spawn_process: at 2015-11-09 13:46:15 running ['/usr/local/bin/lwa28', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '26', '12000000']
2015-11-09 13:46:15 [INFO    ] Finished spawning TBW
2015-11-09 13:46:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-09 13:46:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-09 13:46:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-09 13:46:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-09 13:46:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-09 13:46:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-09 13:46:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-09 13:46:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-09 13:46:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-09 13:46:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-09 13:46:52 [DEBUG   ] Finished updating board flags in 1.220 seconds
2015-11-09 13:47:48 [DEBUG   ] Got command RPT from MCS: ref #528154
2015-11-09 13:47:48 [DEBUG   ] summary = NORMAL
2015-11-09 13:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528154  14 57335 49668547 A NORMALNORMAL'
2015-11-09 13:47:48 [DEBUG   ] Got command RPT from MCS: ref #528155
2015-11-09 13:47:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 13:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528155  48 57335 49668547 A NORMALSystem calibrated and operating normally'
2015-11-09 13:47:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-09 13:47:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-09 13:47:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-09 13:47:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-09 13:47:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-09 13:47:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-09 13:47:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-09 13:47:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-09 13:47:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-09 13:47:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-09 13:47:52 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 13:47:52 [DEBUG   ] Finished updating board flags in 1.296 seconds
2015-11-09 13:48:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-09 13:48:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-09 13:48:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-09 13:48:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-09 13:48:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-09 13:48:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-09 13:48:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-09 13:48:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-09 13:48:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-09 13:48:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-09 13:48:52 [DEBUG   ] Finished updating board flags in 1.218 seconds
2015-11-09 13:49:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-09 13:49:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-09 13:49:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-09 13:49:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-09 13:49:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-09 13:49:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-09 13:49:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-09 13:49:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-09 13:49:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-09 13:49:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-09 13:49:52 [DEBUG   ] Finished updating board flags in 1.115 seconds
2015-11-09 13:49:53 [INFO    ] __waitForTBW - TBW finished in 218.605 s
2015-11-09 13:49:54 [INFO    ] Started antennas statistics background thread
2015-11-09 13:49:54 [INFO    ] Resuming TBN at 38.100 MHz after TBW acquisition
2015-11-09 13:49:54 [DEBUG   ] spawn_process: at 2015-11-09 13:49:54 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '1', '1447076996']
2015-11-09 13:49:54 [DEBUG   ] spawn_process: at 2015-11-09 13:49:54 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '2', '1447076996']
2015-11-09 13:49:54 [DEBUG   ] spawn_process: at 2015-11-09 13:49:54 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '3', '1447076996']
2015-11-09 13:49:54 [DEBUG   ] spawn_process: at 2015-11-09 13:49:54 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '4', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '5', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '6', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '7', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '8', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '9', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '10', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '11', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '12', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '13', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '14', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '15', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '16', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '17', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '18', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '19', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '20', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '21', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '22', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '23', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '24', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '25', '1447076996']
2015-11-09 13:49:55 [DEBUG   ] spawn_process: at 2015-11-09 13:49:55 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '26', '1447076996']
2015-11-09 13:49:55 [INFO    ] Finished spawning TBN in 0.058 s
2015-11-09 13:49:55 [DEBUG   ] Finished updating ant stats in 1.280 seconds
2015-11-09 13:50:07 [DEBUG   ] Got command RPT from MCS: ref #528175
2015-11-09 13:50:07 [DEBUG   ] summary = NORMAL
2015-11-09 13:50:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528175  14 57335 49807747 A NORMALNORMAL'
2015-11-09 13:50:15 [DEBUG   ] Finished updating ant stats in 1.248 seconds
2015-11-09 13:50:35 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 13:50:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-09 13:50:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-09 13:50:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-09 13:50:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-09 13:50:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-09 13:50:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-09 13:50:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-09 13:50:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-09 13:50:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-09 13:50:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-09 13:50:53 [DEBUG   ] Finished updating board flags in 2.193 seconds
2015-11-09 13:50:55 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 13:51:15 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 13:51:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-09 13:51:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-09 13:51:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-09 13:51:53 [DEBUG   ] Finished updating board flags in 2.018 seconds
2015-11-09 13:51:55 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 13:52:15 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 13:52:35 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:52:48 [DEBUG   ] Got command RPT from MCS: ref #528183
2015-11-09 13:52:48 [DEBUG   ] summary = NORMAL
2015-11-09 13:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528183  14 57335 49968543 A NORMALNORMAL'
2015-11-09 13:52:48 [DEBUG   ] Got command RPT from MCS: ref #528184
2015-11-09 13:52:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 13:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528184  48 57335 49968543 A NORMALSystem calibrated and operating normally'
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 13:52:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 13:52:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 13:52:53 [DEBUG   ] Finished updating board flags in 1.928 seconds
2015-11-09 13:52:55 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 13:53:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 13:53:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 13:53:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 13:53:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 13:53:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 13:53:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 13:53:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 13:53:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 13:53:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 13:53:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 13:53:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 13:53:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 13:53:53 [DEBUG   ] Finished updating board flags in 1.811 seconds
2015-11-09 13:53:55 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 13:54:15 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 13:54:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 13:54:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 13:54:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 13:54:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 13:54:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 13:54:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 13:54:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 13:54:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 13:54:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 13:54:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 13:54:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 13:54:53 [DEBUG   ] Finished updating board flags in 1.782 seconds
2015-11-09 13:54:55 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 13:55:07 [DEBUG   ] Got command RPT from MCS: ref #528204
2015-11-09 13:55:07 [DEBUG   ] summary = NORMAL
2015-11-09 13:55:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528204  14 57335 50107827 A NORMALNORMAL'
2015-11-09 13:55:15 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 13:55:36 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-09 13:55:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 13:55:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 13:55:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 13:55:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 13:55:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 13:55:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 13:55:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 13:55:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 13:55:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 13:55:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 13:55:53 [DEBUG   ] Finished updating board flags in 1.612 seconds
2015-11-09 13:55:56 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 13:56:16 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 13:56:36 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 13:56:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 13:56:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 13:56:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 13:56:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 13:56:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 13:56:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 13:56:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 13:56:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 13:56:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 13:56:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 13:56:53 [DEBUG   ] Finished updating board flags in 1.586 seconds
2015-11-09 13:56:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 13:57:16 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 13:57:36 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 13:57:48 [DEBUG   ] Got command RPT from MCS: ref #528213
2015-11-09 13:57:48 [DEBUG   ] summary = NORMAL
2015-11-09 13:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528213  14 57335 50268555 A NORMALNORMAL'
2015-11-09 13:57:48 [DEBUG   ] Got command RPT from MCS: ref #528214
2015-11-09 13:57:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 13:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528214  48 57335 50268555 A NORMALSystem calibrated and operating normally'
2015-11-09 13:57:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 13:57:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 13:57:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 13:57:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 13:57:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 13:57:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 13:57:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 13:57:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 13:57:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 13:57:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 13:57:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 13:57:53 [DEBUG   ] Finished updating board flags in 1.446 seconds
2015-11-09 13:57:56 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-09 13:58:16 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 13:58:36 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 13:58:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 13:58:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 13:58:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 13:58:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 13:58:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 13:58:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 13:58:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 13:58:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 13:58:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 13:58:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 13:58:53 [DEBUG   ] Finished updating board flags in 1.427 seconds
2015-11-09 13:58:56 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 13:59:16 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 13:59:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 13:59:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 13:59:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 13:59:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 13:59:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 13:59:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 13:59:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 13:59:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 13:59:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 13:59:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 13:59:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 13:59:53 [DEBUG   ] Finished updating board flags in 1.425 seconds
2015-11-09 13:59:56 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 14:00:07 [DEBUG   ] Got command RPT from MCS: ref #528233
2015-11-09 14:00:07 [DEBUG   ] summary = NORMAL
2015-11-09 14:00:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528233  14 57335 50407899 A NORMALNORMAL'
2015-11-09 14:00:16 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 14:00:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 14:00:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 14:00:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 14:00:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 14:00:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 14:00:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 14:00:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 14:00:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 14:00:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 14:00:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 14:00:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 14:00:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 14:00:53 [DEBUG   ] Finished updating board flags in 1.422 seconds
2015-11-09 14:00:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 14:01:16 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 14:01:37 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 14:01:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 14:01:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 14:01:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 14:01:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 14:01:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 14:01:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 14:01:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 14:01:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 14:01:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 14:01:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 14:01:53 [DEBUG   ] Finished updating board flags in 1.421 seconds
2015-11-09 14:01:57 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 14:02:17 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 14:02:37 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 14:02:48 [DEBUG   ] Got command RPT from MCS: ref #528237
2015-11-09 14:02:48 [DEBUG   ] summary = NORMAL
2015-11-09 14:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528237  14 57335 50568547 A NORMALNORMAL'
2015-11-09 14:02:48 [DEBUG   ] Got command RPT from MCS: ref #528238
2015-11-09 14:02:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 14:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528238  48 57335 50568547 A NORMALSystem calibrated and operating normally'
2015-11-09 14:02:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 14:02:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 14:02:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 14:02:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 14:02:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 14:02:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 14:02:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 14:02:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 14:02:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 14:02:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 14:02:53 [DEBUG   ] Finished updating board flags in 1.418 seconds
2015-11-09 14:02:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 14:03:17 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 14:03:37 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 14:03:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 14:03:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 14:03:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 14:03:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 14:03:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 14:03:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 14:03:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 14:03:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 14:03:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 14:03:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 14:03:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 14:03:53 [DEBUG   ] Finished updating board flags in 1.549 seconds
2015-11-09 14:03:57 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 14:04:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 14:04:37 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 14:04:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 14:04:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 14:04:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 14:04:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 14:04:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 14:04:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 14:04:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 14:04:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 14:04:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 14:04:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 14:04:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 14:04:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 14:04:54 [DEBUG   ] Finished updating board flags in 2.222 seconds
2015-11-09 14:04:57 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 14:05:07 [DEBUG   ] Got command RPT from MCS: ref #528255
2015-11-09 14:05:07 [DEBUG   ] summary = NORMAL
2015-11-09 14:05:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528255  14 57335 50707985 A NORMALNORMAL'
2015-11-09 14:05:17 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 14:05:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 14:05:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999509
2015-11-09 14:05:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999509
2015-11-09 14:05:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999509
2015-11-09 14:05:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999509
2015-11-09 14:05:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999509
2015-11-09 14:05:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999509
2015-11-09 14:05:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999509
2015-11-09 14:05:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999509
2015-11-09 14:05:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999509
2015-11-09 14:05:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999509
2015-11-09 14:05:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 14:05:54 [DEBUG   ] Finished updating board flags in 2.093 seconds
2015-11-09 14:05:57 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 14:06:17 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 14:06:38 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-09 14:06:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-09 14:06:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-09 14:06:54 [DEBUG   ] Finished updating board flags in 2.028 seconds
2015-11-09 14:06:58 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 14:07:18 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 14:07:38 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 14:07:48 [DEBUG   ] Got command RPT from MCS: ref #528264
2015-11-09 14:07:48 [DEBUG   ] summary = NORMAL
2015-11-09 14:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528264  14 57335 50868551 A NORMALNORMAL'
2015-11-09 14:07:48 [DEBUG   ] Got command RPT from MCS: ref #528265
2015-11-09 14:07:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 14:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528265  48 57335 50868551 A NORMALSystem calibrated and operating normally'
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 14:07:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 14:07:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 14:07:54 [DEBUG   ] Finished updating board flags in 1.989 seconds
2015-11-09 14:07:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 14:08:18 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 14:08:38 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2015-11-09 14:08:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:08:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:08:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:08:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:08:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:08:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:08:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:08:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:08:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:08:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:08:54 [DEBUG   ] Finished updating board flags in 1.806 seconds
2015-11-09 14:08:58 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-09 14:09:18 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 14:09:38 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 14:09:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:09:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:09:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:09:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:09:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:09:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:09:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:09:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:09:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:09:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:09:54 [DEBUG   ] Finished updating board flags in 1.790 seconds
2015-11-09 14:09:58 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 14:10:08 [DEBUG   ] Got command RPT from MCS: ref #528284
2015-11-09 14:10:08 [DEBUG   ] summary = NORMAL
2015-11-09 14:10:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528284  14 57335 51008051 A NORMALNORMAL'
2015-11-09 14:10:18 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2015-11-09 14:10:39 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-09 14:10:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:10:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:10:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:10:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:10:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:10:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:10:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:10:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:10:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:10:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:10:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 14:10:54 [DEBUG   ] Finished updating board flags in 1.613 seconds
2015-11-09 14:10:59 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 14:11:19 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 14:11:39 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 14:11:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:11:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:11:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:11:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:11:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:11:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:11:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:11:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:11:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:11:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:11:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 14:11:54 [DEBUG   ] Finished updating board flags in 1.582 seconds
2015-11-09 14:11:59 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 14:12:19 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 14:12:39 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 14:12:48 [DEBUG   ] Got command RPT from MCS: ref #528288
2015-11-09 14:12:48 [DEBUG   ] summary = NORMAL
2015-11-09 14:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528288  14 57335 51168547 A NORMALNORMAL'
2015-11-09 14:12:48 [DEBUG   ] Got command RPT from MCS: ref #528289
2015-11-09 14:12:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 14:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528289  48 57335 51168547 A NORMALSystem calibrated and operating normally'
2015-11-09 14:12:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:12:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:12:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:12:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:12:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:12:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 14:12:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:12:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:12:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:12:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:12:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:12:54 [DEBUG   ] Finished updating board flags in 1.415 seconds
2015-11-09 14:12:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 14:13:19 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 14:13:39 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 14:13:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:13:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:13:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:13:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:13:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:13:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:13:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:13:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:13:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:13:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:13:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 14:13:54 [DEBUG   ] Finished updating board flags in 1.409 seconds
2015-11-09 14:14:00 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-09 14:14:20 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 14:14:40 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 14:14:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:14:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:14:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:14:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:14:53 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:14:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:14:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:14:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:14:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:14:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:14:54 [DEBUG   ] Finished updating board flags in 1.386 seconds
2015-11-09 14:15:00 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 14:15:08 [DEBUG   ] Got command RPT from MCS: ref #528309
2015-11-09 14:15:08 [DEBUG   ] summary = NORMAL
2015-11-09 14:15:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528309  14 57335 51308139 A NORMALNORMAL'
2015-11-09 14:15:20 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 14:15:40 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 14:15:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:15:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:15:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:15:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:15:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:15:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:15:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:15:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:15:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:15:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:15:55 [DEBUG   ] Finished updating board flags in 2.220 seconds
2015-11-09 14:16:00 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 14:16:20 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 14:16:40 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 14:16:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:16:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:16:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:16:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:16:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:16:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:16:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:16:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:16:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:16:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:16:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 14:16:55 [DEBUG   ] Finished updating board flags in 2.186 seconds
2015-11-09 14:17:00 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2015-11-09 14:17:20 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 14:17:40 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 14:17:48 [DEBUG   ] Got command RPT from MCS: ref #528318
2015-11-09 14:17:48 [DEBUG   ] summary = NORMAL
2015-11-09 14:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528318  14 57335 51468559 A NORMALNORMAL'
2015-11-09 14:17:48 [DEBUG   ] Got command RPT from MCS: ref #528319
2015-11-09 14:17:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 14:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528319  48 57335 51468559 A NORMALSystem calibrated and operating normally'
2015-11-09 14:17:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:17:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:17:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:17:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:17:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:17:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:17:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:17:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:17:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:17:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:17:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 14:17:55 [DEBUG   ] Finished updating board flags in 2.009 seconds
2015-11-09 14:18:00 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 14:18:20 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 14:18:41 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-09 14:18:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:18:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:18:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:18:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:18:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:18:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:18:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:18:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:18:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:18:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:18:55 [DEBUG   ] Finished updating board flags in 1.998 seconds
2015-11-09 14:19:01 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 14:19:21 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 14:19:41 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 14:19:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:19:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:19:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:19:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:19:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:19:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 14:19:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:19:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:19:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:19:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:19:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:19:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 14:19:55 [DEBUG   ] Finished updating board flags in 1.794 seconds
2015-11-09 14:20:01 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2015-11-09 14:20:08 [DEBUG   ] Got command RPT from MCS: ref #528335
2015-11-09 14:20:08 [DEBUG   ] summary = NORMAL
2015-11-09 14:20:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528335  14 57335 51608219 A NORMALNORMAL'
2015-11-09 14:20:21 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 14:20:41 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 14:20:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:20:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:20:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:20:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:20:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:20:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:20:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:20:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:20:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:20:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:20:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 14:20:55 [DEBUG   ] Finished updating board flags in 1.617 seconds
2015-11-09 14:21:01 [DEBUG   ] Finished updating ant stats in 0.807 seconds
2015-11-09 14:21:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 14:21:41 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 14:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:21:55 [DEBUG   ] Finished updating board flags in 1.578 seconds
2015-11-09 14:22:01 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 14:22:21 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 14:22:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 14:22:48 [DEBUG   ] Got command RPT from MCS: ref #528339
2015-11-09 14:22:48 [DEBUG   ] summary = NORMAL
2015-11-09 14:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528339  14 57335 51768555 A NORMALNORMAL'
2015-11-09 14:22:48 [DEBUG   ] Got command RPT from MCS: ref #528340
2015-11-09 14:22:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 14:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528340  48 57335 51768555 A NORMALSystem calibrated and operating normally'
2015-11-09 14:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:22:55 [DEBUG   ] Finished updating board flags in 1.419 seconds
2015-11-09 14:23:01 [DEBUG   ] Finished updating ant stats in 0.537 seconds
2015-11-09 14:23:22 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2015-11-09 14:23:42 [DEBUG   ] Finished updating ant stats in 1.210 seconds
2015-11-09 14:23:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:23:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:23:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:23:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:23:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:23:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:23:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:23:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:23:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:23:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:23:55 [DEBUG   ] Finished updating board flags in 1.402 seconds
2015-11-09 14:24:02 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 14:24:22 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 14:24:42 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 14:24:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:24:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:24:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:24:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:24:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:24:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:24:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:24:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:24:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:24:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:24:55 [DEBUG   ] Finished updating board flags in 1.409 seconds
2015-11-09 14:25:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 14:25:08 [DEBUG   ] Got command RPT from MCS: ref #528360
2015-11-09 14:25:08 [DEBUG   ] summary = NORMAL
2015-11-09 14:25:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528360  14 57335 51908303 A NORMALNORMAL'
2015-11-09 14:25:22 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 14:25:42 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 14:25:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:25:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:25:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:25:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:25:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:25:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:25:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:25:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:25:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:25:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:25:55 [DEBUG   ] Finished updating board flags in 1.442 seconds
2015-11-09 14:26:02 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-09 14:26:22 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 14:26:42 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 14:26:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:26:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:26:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:26:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:26:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:26:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:26:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:26:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:26:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:26:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:26:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 14:26:55 [DEBUG   ] Finished updating board flags in 1.446 seconds
2015-11-09 14:27:02 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 14:27:22 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 14:27:42 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 14:27:48 [DEBUG   ] Got command RPT from MCS: ref #528369
2015-11-09 14:27:48 [DEBUG   ] summary = NORMAL
2015-11-09 14:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528369  14 57335 52068551 A NORMALNORMAL'
2015-11-09 14:27:48 [DEBUG   ] Got command RPT from MCS: ref #528370
2015-11-09 14:27:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 14:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528370  48 57335 52068551 A NORMALSystem calibrated and operating normally'
2015-11-09 14:27:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:27:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:27:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:27:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:27:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:27:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 14:27:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:27:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:27:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:27:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:27:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:27:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 14:27:55 [DEBUG   ] Finished updating board flags in 1.459 seconds
2015-11-09 14:28:02 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 14:28:22 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 14:28:42 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2015-11-09 14:28:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:28:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:28:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:28:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:28:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:28:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:28:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:28:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:28:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:28:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:28:55 [DEBUG   ] Finished updating board flags in 1.450 seconds
2015-11-09 14:29:02 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 14:29:22 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 14:29:42 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 14:29:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:29:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:29:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:29:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:29:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:29:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:29:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:29:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:29:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:29:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:29:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 14:29:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 14:29:56 [DEBUG   ] Finished updating board flags in 2.255 seconds
2015-11-09 14:30:03 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 14:30:08 [DEBUG   ] Got command RPT from MCS: ref #528389
2015-11-09 14:30:08 [DEBUG   ] summary = NORMAL
2015-11-09 14:30:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528389  14 57335 52208387 A NORMALNORMAL'
2015-11-09 14:30:23 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 14:30:43 [DEBUG   ] Finished updating ant stats in 0.911 seconds
2015-11-09 14:30:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:30:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:30:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:30:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:30:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:30:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:30:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:30:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:30:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:30:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:30:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 14:30:56 [DEBUG   ] Finished updating board flags in 1.647 seconds
2015-11-09 14:31:03 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2015-11-09 14:31:23 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 14:31:43 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 14:31:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:31:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:31:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:31:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:31:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:31:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:31:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:31:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:31:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:31:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:31:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 14:31:56 [DEBUG   ] Finished updating board flags in 1.613 seconds
2015-11-09 14:32:03 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 14:32:24 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-09 14:32:44 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 14:32:48 [DEBUG   ] Got command RPT from MCS: ref #528393
2015-11-09 14:32:48 [DEBUG   ] summary = NORMAL
2015-11-09 14:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528393  14 57335 52368551 A NORMALNORMAL'
2015-11-09 14:32:48 [DEBUG   ] Got command RPT from MCS: ref #528394
2015-11-09 14:32:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 14:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528394  48 57335 52368551 A NORMALSystem calibrated and operating normally'
2015-11-09 14:32:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:32:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:32:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:32:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:32:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:32:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:32:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:32:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:32:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:32:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:32:56 [DEBUG   ] Finished updating board flags in 1.609 seconds
2015-11-09 14:33:04 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 14:33:24 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 14:33:44 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 14:33:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:33:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:33:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:33:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:33:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:33:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:33:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:33:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:33:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:33:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:33:56 [DEBUG   ] Finished updating board flags in 1.584 seconds
2015-11-09 14:34:04 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 14:34:24 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 14:34:44 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 14:34:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:34:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:34:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:34:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:34:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:34:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:34:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:34:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:34:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:34:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:34:56 [DEBUG   ] Finished updating board flags in 1.479 seconds
2015-11-09 14:35:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 14:35:08 [DEBUG   ] Got command RPT from MCS: ref #528411
2015-11-09 14:35:08 [DEBUG   ] summary = NORMAL
2015-11-09 14:35:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528411  14 57335 52508463 A NORMALNORMAL'
2015-11-09 14:35:24 [DEBUG   ] Finished updating ant stats in 1.195 seconds
2015-11-09 14:35:44 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 14:35:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:35:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:35:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:35:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:35:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:35:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:35:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:35:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:35:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:35:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:35:56 [DEBUG   ] Finished updating board flags in 1.477 seconds
2015-11-09 14:36:04 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 14:36:24 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-09 14:36:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 14:36:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:36:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:36:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:36:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:36:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:36:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:36:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:36:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:36:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:36:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:36:56 [DEBUG   ] Finished updating board flags in 1.486 seconds
2015-11-09 14:37:04 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 14:37:24 [DEBUG   ] Finished updating ant stats in 0.593 seconds
2015-11-09 14:37:45 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2015-11-09 14:37:48 [DEBUG   ] Got command RPT from MCS: ref #528420
2015-11-09 14:37:48 [DEBUG   ] summary = NORMAL
2015-11-09 14:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528420  14 57335 52668563 A NORMALNORMAL'
2015-11-09 14:37:48 [DEBUG   ] Got command RPT from MCS: ref #528421
2015-11-09 14:37:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 14:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528421  48 57335 52668563 A NORMALSystem calibrated and operating normally'
2015-11-09 14:37:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:37:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:37:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:37:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:37:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:37:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:37:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:37:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:37:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:37:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:37:57 [DEBUG   ] Finished updating board flags in 2.162 seconds
2015-11-09 14:38:05 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 14:38:25 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 14:38:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 14:38:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:38:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:38:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:38:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:38:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:38:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:38:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:38:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:38:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:38:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:38:57 [DEBUG   ] Finished updating board flags in 1.976 seconds
2015-11-09 14:39:05 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 14:39:25 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 14:39:45 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 14:39:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:39:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:39:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:39:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:39:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:39:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:39:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:39:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:39:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:39:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:39:57 [DEBUG   ] Finished updating board flags in 1.805 seconds
2015-11-09 14:40:05 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 14:40:08 [DEBUG   ] Got command RPT from MCS: ref #528440
2015-11-09 14:40:08 [DEBUG   ] summary = NORMAL
2015-11-09 14:40:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528440  14 57335 52808543 A NORMALNORMAL'
2015-11-09 14:40:25 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 14:40:45 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 14:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:40:57 [DEBUG   ] Finished updating board flags in 1.791 seconds
2015-11-09 14:41:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 14:41:25 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 14:41:45 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 14:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:41:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 14:41:57 [DEBUG   ] Finished updating board flags in 1.625 seconds
2015-11-09 14:42:05 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2015-11-09 14:42:25 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 14:42:45 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 14:42:48 [DEBUG   ] Got command RPT from MCS: ref #528444
2015-11-09 14:42:48 [DEBUG   ] summary = NORMAL
2015-11-09 14:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528444  14 57335 52968555 A NORMALNORMAL'
2015-11-09 14:42:48 [DEBUG   ] Got command RPT from MCS: ref #528445
2015-11-09 14:42:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 14:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528445  48 57335 52968555 A NORMALSystem calibrated and operating normally'
2015-11-09 14:42:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:42:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:42:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:42:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:42:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:42:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:42:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:42:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:42:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:42:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:42:57 [DEBUG   ] Finished updating board flags in 1.594 seconds
2015-11-09 14:43:05 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 14:43:26 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-09 14:43:46 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 14:43:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:43:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:43:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:43:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:43:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:43:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:43:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:43:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:43:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:43:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:43:57 [DEBUG   ] Finished updating board flags in 1.401 seconds
2015-11-09 14:44:06 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 14:44:26 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 14:44:46 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 14:44:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:44:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:44:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:44:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:44:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:44:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:44:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:44:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:44:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:44:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:44:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 14:44:57 [DEBUG   ] Finished updating board flags in 1.403 seconds
2015-11-09 14:45:06 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 14:45:08 [DEBUG   ] Got command RPT from MCS: ref #528465
2015-11-09 14:45:08 [DEBUG   ] summary = NORMAL
2015-11-09 14:45:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528465  14 57335 53108647 A NORMALNORMAL'
2015-11-09 14:45:26 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 14:45:46 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 14:45:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:45:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:45:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:45:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:45:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:45:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:45:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:45:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:45:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:45:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:45:57 [DEBUG   ] Finished updating board flags in 1.384 seconds
2015-11-09 14:46:06 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2015-11-09 14:46:26 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 14:46:46 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 14:46:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:46:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:46:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:46:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:46:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:46:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:46:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:46:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:46:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:46:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:46:57 [DEBUG   ] Finished updating board flags in 1.415 seconds
2015-11-09 14:47:06 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 14:47:26 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 14:47:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 14:47:48 [DEBUG   ] Got command RPT from MCS: ref #528474
2015-11-09 14:47:48 [DEBUG   ] summary = NORMAL
2015-11-09 14:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528474  14 57335 53268563 A NORMALNORMAL'
2015-11-09 14:47:48 [DEBUG   ] Got command RPT from MCS: ref #528475
2015-11-09 14:47:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 14:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528475  48 57335 53268563 A NORMALSystem calibrated and operating normally'
2015-11-09 14:47:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:47:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:47:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:47:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:47:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:47:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:47:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:47:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:47:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:47:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:47:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 14:47:58 [DEBUG   ] Finished updating board flags in 2.244 seconds
2015-11-09 14:48:06 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 14:48:26 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 14:48:46 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 14:48:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:48:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:48:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:48:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:48:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:48:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:48:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:48:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:48:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:48:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:48:58 [DEBUG   ] Finished updating board flags in 2.149 seconds
2015-11-09 14:49:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 14:49:26 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 14:49:47 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 14:49:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:49:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:49:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:49:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:49:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:49:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:49:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:49:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:49:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:49:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:49:58 [DEBUG   ] Finished updating board flags in 2.018 seconds
2015-11-09 14:50:07 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 14:50:08 [DEBUG   ] Got command RPT from MCS: ref #528491
2015-11-09 14:50:08 [DEBUG   ] summary = NORMAL
2015-11-09 14:50:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528491  14 57335 53408739 A NORMALNORMAL'
2015-11-09 14:50:27 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 14:50:47 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 14:50:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:50:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:50:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:50:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:50:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:50:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:50:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:50:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:50:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:50:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:50:58 [DEBUG   ] Finished updating board flags in 1.988 seconds
2015-11-09 14:51:07 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-09 14:51:27 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 14:51:47 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 14:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:51:58 [DEBUG   ] Finished updating board flags in 1.807 seconds
2015-11-09 14:52:07 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 14:52:27 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-09 14:52:47 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 14:52:48 [DEBUG   ] Got command RPT from MCS: ref #528495
2015-11-09 14:52:48 [DEBUG   ] summary = NORMAL
2015-11-09 14:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528495  14 57335 53568555 A NORMALNORMAL'
2015-11-09 14:52:48 [DEBUG   ] Got command RPT from MCS: ref #528496
2015-11-09 14:52:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 14:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528496  48 57335 53568555 A NORMALSystem calibrated and operating normally'
2015-11-09 14:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:52:58 [DEBUG   ] Finished updating board flags in 1.796 seconds
2015-11-09 14:53:07 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 14:53:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 14:53:48 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 14:53:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:53:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:53:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:53:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:53:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:53:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:53:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:53:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:53:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:53:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:53:58 [DEBUG   ] Finished updating board flags in 1.610 seconds
2015-11-09 14:54:08 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 14:54:28 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 14:54:48 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 14:54:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:54:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:54:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:54:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:54:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:54:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:54:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:54:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:54:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:54:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:54:58 [DEBUG   ] Finished updating board flags in 1.608 seconds
2015-11-09 14:55:08 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 14:55:08 [DEBUG   ] Got command RPT from MCS: ref #528516
2015-11-09 14:55:08 [DEBUG   ] summary = NORMAL
2015-11-09 14:55:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528516  14 57335 53708815 A NORMALNORMAL'
2015-11-09 14:55:28 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 14:55:48 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 14:55:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:55:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:55:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:55:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:55:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:55:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:55:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:55:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:55:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:55:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:55:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 14:55:58 [DEBUG   ] Finished updating board flags in 1.584 seconds
2015-11-09 14:56:08 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 14:56:28 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 14:56:48 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 14:56:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:56:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:56:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:56:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:56:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:56:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:56:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:56:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:56:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:56:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:56:58 [DEBUG   ] Finished updating board flags in 1.420 seconds
2015-11-09 14:57:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 14:57:28 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 14:57:48 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 14:57:48 [DEBUG   ] Got command RPT from MCS: ref #528525
2015-11-09 14:57:48 [DEBUG   ] summary = NORMAL
2015-11-09 14:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528525  14 57335 53868563 A NORMALNORMAL'
2015-11-09 14:57:48 [DEBUG   ] Got command RPT from MCS: ref #528526
2015-11-09 14:57:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 14:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528526  48 57335 53868563 A NORMALSystem calibrated and operating normally'
2015-11-09 14:57:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:57:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:57:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:57:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:57:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:57:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:57:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:57:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:57:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:57:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:57:58 [DEBUG   ] Finished updating board flags in 1.485 seconds
2015-11-09 14:58:08 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 14:58:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 14:58:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 14:58:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:58:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:58:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:58:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:58:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:58:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:58:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:58:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:58:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:58:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:58:58 [DEBUG   ] Finished updating board flags in 1.494 seconds
2015-11-09 14:59:08 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 14:59:29 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-09 14:59:49 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 14:59:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 14:59:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 14:59:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 14:59:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 14:59:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 14:59:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 14:59:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 14:59:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 14:59:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 14:59:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 14:59:58 [DEBUG   ] Finished updating board flags in 1.486 seconds
2015-11-09 15:00:08 [DEBUG   ] Got command RPT from MCS: ref #528545
2015-11-09 15:00:08 [DEBUG   ] summary = NORMAL
2015-11-09 15:00:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528545  14 57335 54008903 A NORMALNORMAL'
2015-11-09 15:00:09 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-09 15:00:29 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 15:00:49 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 15:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:00:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:00:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:00:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:00:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:00:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:00:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 15:00:59 [DEBUG   ] Finished updating board flags in 2.216 seconds
2015-11-09 15:01:09 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 15:01:29 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 15:01:49 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 15:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:01:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:01:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:01:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:01:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:01:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:01:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 15:01:59 [DEBUG   ] Finished updating board flags in 2.211 seconds
2015-11-09 15:02:09 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2015-11-09 15:02:29 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 15:02:48 [DEBUG   ] Got command RPT from MCS: ref #528549
2015-11-09 15:02:48 [DEBUG   ] summary = NORMAL
2015-11-09 15:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528549  14 57335 54168563 A NORMALNORMAL'
2015-11-09 15:02:48 [DEBUG   ] Got command RPT from MCS: ref #528550
2015-11-09 15:02:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 15:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528550  48 57335 54168564 A NORMALSystem calibrated and operating normally'
2015-11-09 15:02:49 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2015-11-09 15:02:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:02:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:02:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:02:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:02:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:02:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:02:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:02:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:02:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:02:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:02:59 [DEBUG   ] Finished updating board flags in 2.100 seconds
2015-11-09 15:03:09 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 15:03:29 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 15:03:49 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 15:03:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:03:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:03:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:03:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:03:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:03:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:03:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:03:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:03:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:03:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:03:59 [DEBUG   ] Finished updating board flags in 2.021 seconds
2015-11-09 15:04:09 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 15:04:29 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 15:04:49 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2015-11-09 15:04:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:04:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:04:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:04:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:04:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:04:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:04:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:04:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:04:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:04:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:04:59 [DEBUG   ] Finished updating board flags in 2.004 seconds
2015-11-09 15:05:08 [DEBUG   ] Got command RPT from MCS: ref #528567
2015-11-09 15:05:08 [DEBUG   ] summary = NORMAL
2015-11-09 15:05:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528567  14 57335 54308983 A NORMALNORMAL'
2015-11-09 15:05:09 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 15:05:29 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 15:05:50 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-09 15:05:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:05:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:05:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:05:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:05:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:05:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:05:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:05:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:05:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:05:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:05:59 [DEBUG   ] Finished updating board flags in 1.995 seconds
2015-11-09 15:06:10 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 15:06:30 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 15:06:50 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 15:06:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:06:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:06:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:06:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:06:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:06:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:06:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:06:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:06:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:06:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:06:59 [DEBUG   ] Finished updating board flags in 1.804 seconds
2015-11-09 15:07:10 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 15:07:30 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2015-11-09 15:07:48 [DEBUG   ] Got command RPT from MCS: ref #528576
2015-11-09 15:07:48 [DEBUG   ] summary = NORMAL
2015-11-09 15:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528576  14 57335 54468559 A NORMALNORMAL'
2015-11-09 15:07:48 [DEBUG   ] Got command RPT from MCS: ref #528577
2015-11-09 15:07:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 15:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528577  48 57335 54468559 A NORMALSystem calibrated and operating normally'
2015-11-09 15:07:50 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 15:07:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:07:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:07:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:07:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:07:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:07:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:07:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:07:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:07:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:07:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:07:59 [DEBUG   ] Finished updating board flags in 1.805 seconds
2015-11-09 15:08:10 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 15:08:30 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 15:08:50 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 15:08:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:08:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:08:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:08:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:08:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:08:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:08:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:08:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:08:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:08:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:08:59 [DEBUG   ] Finished updating board flags in 1.789 seconds
2015-11-09 15:09:10 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 15:09:30 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 15:09:50 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 15:09:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:09:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:09:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:09:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:09:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:09:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:09:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:09:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:09:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:09:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:09:59 [DEBUG   ] Finished updating board flags in 1.618 seconds
2015-11-09 15:10:09 [DEBUG   ] Got command RPT from MCS: ref #528596
2015-11-09 15:10:09 [DEBUG   ] summary = NORMAL
2015-11-09 15:10:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528596  14 57335 54609075 A NORMALNORMAL'
2015-11-09 15:10:10 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 15:10:30 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 15:10:51 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-09 15:10:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:10:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:10:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:10:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:10:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:10:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 15:10:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:10:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:10:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:10:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:10:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:10:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 15:10:59 [DEBUG   ] Finished updating board flags in 1.586 seconds
2015-11-09 15:11:11 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 15:11:31 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 15:11:51 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 15:11:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:11:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:11:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:11:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:11:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:11:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:11:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:11:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:11:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:11:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:11:59 [DEBUG   ] Finished updating board flags in 1.414 seconds
2015-11-09 15:12:11 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 15:12:31 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 15:12:48 [DEBUG   ] Got command RPT from MCS: ref #528600
2015-11-09 15:12:48 [DEBUG   ] summary = NORMAL
2015-11-09 15:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528600  14 57335 54768559 A NORMALNORMAL'
2015-11-09 15:12:48 [DEBUG   ] Got command RPT from MCS: ref #528601
2015-11-09 15:12:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 15:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528601  48 57335 54768559 A NORMALSystem calibrated and operating normally'
2015-11-09 15:12:51 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 15:12:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:12:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:12:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:12:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:12:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:12:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:12:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:12:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:12:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:12:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:12:59 [DEBUG   ] Finished updating board flags in 1.486 seconds
2015-11-09 15:13:11 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 15:13:31 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 15:13:51 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 15:13:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:13:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:13:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:13:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:13:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:13:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:13:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:13:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:13:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:13:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:13:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 15:13:59 [DEBUG   ] Finished updating board flags in 1.480 seconds
2015-11-09 15:14:11 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2015-11-09 15:14:31 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 15:14:51 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 15:14:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:14:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:14:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:14:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:14:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:14:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:14:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:14:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:14:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:14:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:14:59 [DEBUG   ] Finished updating board flags in 1.484 seconds
2015-11-09 15:15:09 [DEBUG   ] Got command RPT from MCS: ref #528624
2015-11-09 15:15:09 [DEBUG   ] summary = NORMAL
2015-11-09 15:15:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528624  14 57335 54909155 A NORMALNORMAL'
2015-11-09 15:15:11 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 15:15:31 [DEBUG   ] Finished updating ant stats in 0.795 seconds
2015-11-09 15:15:51 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 15:15:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:15:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:15:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:15:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:15:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:16:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:16:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:16:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:16:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:16:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:16:00 [DEBUG   ] Finished updating board flags in 2.225 seconds
2015-11-09 15:16:11 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 15:16:31 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 15:16:51 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 15:16:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:16:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:16:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:16:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:16:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:17:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:17:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:17:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:17:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:17:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:17:00 [DEBUG   ] Finished updating board flags in 2.229 seconds
2015-11-09 15:17:12 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 15:17:32 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-09 15:17:48 [DEBUG   ] Got command RPT from MCS: ref #528635
2015-11-09 15:17:48 [DEBUG   ] summary = NORMAL
2015-11-09 15:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528635  14 57335 55068563 A NORMALNORMAL'
2015-11-09 15:17:48 [DEBUG   ] Got command RPT from MCS: ref #528636
2015-11-09 15:17:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 15:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528636  48 57335 55068563 A NORMALSystem calibrated and operating normally'
2015-11-09 15:17:52 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 15:17:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:17:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:17:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:17:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:17:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:18:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:18:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:18:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:18:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:18:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:18:00 [DEBUG   ] Finished updating board flags in 2.088 seconds
2015-11-09 15:18:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 15:18:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 15:18:52 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 15:18:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:18:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:18:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:18:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:18:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:19:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:19:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:19:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:19:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:19:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:19:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 15:19:00 [DEBUG   ] Finished updating board flags in 2.016 seconds
2015-11-09 15:19:12 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 15:19:32 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 15:19:52 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 15:19:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:19:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:19:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:19:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:19:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:20:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:20:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:20:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:20:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:20:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:20:00 [DEBUG   ] Finished updating board flags in 1.985 seconds
2015-11-09 15:20:09 [DEBUG   ] Got command RPT from MCS: ref #528652
2015-11-09 15:20:09 [DEBUG   ] summary = NORMAL
2015-11-09 15:20:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528652  14 57335 55209231 A NORMALNORMAL'
2015-11-09 15:20:12 [DEBUG   ] Finished updating ant stats in 0.991 seconds
2015-11-09 15:20:32 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 15:20:52 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2015-11-09 15:20:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:20:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:20:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:20:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:20:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:21:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:21:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:21:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:21:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:21:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:21:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 15:21:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 15:21:00 [DEBUG   ] Finished updating board flags in 1.809 seconds
2015-11-09 15:21:12 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 15:21:32 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 15:21:53 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 15:21:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:21:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:21:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:21:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:21:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:22:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:22:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:22:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:22:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:22:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:22:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 15:22:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 15:22:00 [DEBUG   ] Finished updating board flags in 1.809 seconds
2015-11-09 15:22:13 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 15:22:33 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 15:22:48 [DEBUG   ] Got command RPT from MCS: ref #528656
2015-11-09 15:22:48 [DEBUG   ] summary = NORMAL
2015-11-09 15:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528656  14 57335 55368563 A NORMALNORMAL'
2015-11-09 15:22:48 [DEBUG   ] Got command RPT from MCS: ref #528657
2015-11-09 15:22:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 15:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528657  48 57335 55368564 A NORMALSystem calibrated and operating normally'
2015-11-09 15:22:53 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 15:22:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:22:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:22:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:22:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:22:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:23:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:23:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:23:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:23:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:23:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:23:00 [DEBUG   ] Finished updating board flags in 1.786 seconds
2015-11-09 15:23:13 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 15:23:33 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 15:23:53 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 15:23:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:23:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:23:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:23:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:23:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:24:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:24:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:24:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:24:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:24:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:24:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 15:24:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 15:24:00 [DEBUG   ] Finished updating board flags in 1.608 seconds
2015-11-09 15:24:13 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 15:24:33 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-09 15:24:53 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 15:24:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:24:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:24:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:24:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:24:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:25:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:25:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:25:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:25:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:25:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:25:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 15:25:00 [DEBUG   ] Finished updating board flags in 1.610 seconds
2015-11-09 15:25:09 [DEBUG   ] Got command RPT from MCS: ref #528677
2015-11-09 15:25:09 [DEBUG   ] summary = NORMAL
2015-11-09 15:25:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528677  14 57335 55509323 A NORMALNORMAL'
2015-11-09 15:25:13 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 15:25:33 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 15:25:53 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 15:25:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:25:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:25:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:25:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:25:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:26:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:26:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:26:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:26:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:26:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:26:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 15:26:00 [DEBUG   ] Finished updating board flags in 1.582 seconds
2015-11-09 15:26:13 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 15:26:33 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 15:26:54 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 15:26:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:26:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:26:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:26:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:26:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:27:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:27:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:27:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:27:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:27:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:27:00 [DEBUG   ] Finished updating board flags in 1.420 seconds
2015-11-09 15:27:14 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 15:27:34 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 15:27:48 [DEBUG   ] Got command RPT from MCS: ref #528686
2015-11-09 15:27:48 [DEBUG   ] summary = NORMAL
2015-11-09 15:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528686  14 57335 55668563 A NORMALNORMAL'
2015-11-09 15:27:48 [DEBUG   ] Got command RPT from MCS: ref #528687
2015-11-09 15:27:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 15:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528687  48 57335 55668563 A NORMALSystem calibrated and operating normally'
2015-11-09 15:27:54 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 15:27:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:27:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:27:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:27:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:27:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:27:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 15:28:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:28:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:28:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:28:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:28:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:28:00 [DEBUG   ] Finished updating board flags in 1.394 seconds
2015-11-09 15:28:14 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 15:28:34 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 15:28:54 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 15:29:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:29:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:29:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:29:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:29:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:29:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:29:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:29:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:29:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:29:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:29:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 15:29:01 [DEBUG   ] Finished updating board flags in 2.209 seconds
2015-11-09 15:29:14 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 15:29:34 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 15:29:54 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 15:30:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:30:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:30:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:30:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:30:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:30:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:30:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:30:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:30:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:30:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:30:01 [DEBUG   ] Finished updating board flags in 2.190 seconds
2015-11-09 15:30:09 [DEBUG   ] Got command RPT from MCS: ref #528706
2015-11-09 15:30:09 [DEBUG   ] summary = NORMAL
2015-11-09 15:30:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528706  14 57335 55809403 A NORMALNORMAL'
2015-11-09 15:30:14 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 15:30:34 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 15:30:54 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 15:31:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:31:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:31:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:31:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:31:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:31:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:31:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:31:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:31:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:31:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:31:01 [DEBUG   ] Finished updating board flags in 2.019 seconds
2015-11-09 15:31:14 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 15:31:34 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 15:31:54 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 15:32:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:32:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:32:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:32:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:32:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:32:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:32:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:32:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:32:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:32:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:32:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 15:32:01 [DEBUG   ] Finished updating board flags in 1.980 seconds
2015-11-09 15:32:14 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 15:32:34 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 15:32:48 [DEBUG   ] Got command RPT from MCS: ref #528710
2015-11-09 15:32:48 [DEBUG   ] summary = NORMAL
2015-11-09 15:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528710  14 57335 55968567 A NORMALNORMAL'
2015-11-09 15:32:48 [DEBUG   ] Got command RPT from MCS: ref #528711
2015-11-09 15:32:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 15:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528711  48 57335 55968567 A NORMALSystem calibrated and operating normally'
2015-11-09 15:32:54 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 15:33:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:33:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:33:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:33:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:33:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:33:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:33:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:33:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:33:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:33:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:33:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 15:33:01 [DEBUG   ] Finished updating board flags in 1.828 seconds
2015-11-09 15:33:15 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 15:33:35 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 15:33:55 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 15:34:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:34:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:34:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:34:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:34:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:34:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:34:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:34:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:34:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:34:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:34:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 15:34:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 15:34:01 [DEBUG   ] Finished updating board flags in 1.792 seconds
2015-11-09 15:34:15 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 15:34:35 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 15:34:55 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 15:35:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:35:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:35:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:35:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:35:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:35:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 15:35:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:35:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:35:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:35:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:35:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:35:01 [DEBUG   ] Finished updating board flags in 1.603 seconds
2015-11-09 15:35:09 [DEBUG   ] Got command RPT from MCS: ref #528728
2015-11-09 15:35:09 [DEBUG   ] summary = NORMAL
2015-11-09 15:35:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528728  14 57335 56109487 A NORMALNORMAL'
2015-11-09 15:35:15 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 15:35:35 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 15:35:55 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2015-11-09 15:36:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:36:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:36:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:36:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:36:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:36:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:36:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:36:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:36:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:36:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:36:01 [DEBUG   ] Finished updating board flags in 1.610 seconds
2015-11-09 15:36:15 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 15:36:35 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 15:36:55 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 15:37:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:37:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:37:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:37:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:37:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:37:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:37:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:37:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:37:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:37:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:37:01 [DEBUG   ] Finished updating board flags in 1.579 seconds
2015-11-09 15:37:15 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-09 15:37:35 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 15:37:48 [DEBUG   ] Got command RPT from MCS: ref #528737
2015-11-09 15:37:48 [DEBUG   ] summary = NORMAL
2015-11-09 15:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528737  14 57335 56268571 A NORMALNORMAL'
2015-11-09 15:37:48 [DEBUG   ] Got command RPT from MCS: ref #528738
2015-11-09 15:37:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 15:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528738  48 57335 56268571 A NORMALSystem calibrated and operating normally'
2015-11-09 15:37:55 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 15:38:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:38:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:38:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:38:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:38:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:38:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:38:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:38:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:38:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:38:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:38:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 15:38:01 [DEBUG   ] Finished updating board flags in 1.426 seconds
2015-11-09 15:38:16 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 15:38:36 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 15:38:56 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 15:39:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:39:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:39:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:39:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:39:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:39:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:39:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:39:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:39:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:39:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:39:01 [DEBUG   ] Finished updating board flags in 1.392 seconds
2015-11-09 15:39:16 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-09 15:39:36 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 15:39:56 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 15:40:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:40:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:40:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:40:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:40:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:40:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:40:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:40:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:40:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:40:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:40:02 [DEBUG   ] Finished updating board flags in 2.217 seconds
2015-11-09 15:40:09 [DEBUG   ] Got command RPT from MCS: ref #528757
2015-11-09 15:40:09 [DEBUG   ] summary = NORMAL
2015-11-09 15:40:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528757  14 57335 56409559 A NORMALNORMAL'
2015-11-09 15:40:16 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 15:40:36 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 15:40:56 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 15:41:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:41:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:41:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:41:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:41:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:41:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 15:41:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:41:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:41:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:41:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:41:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:41:02 [DEBUG   ] Finished updating board flags in 2.184 seconds
2015-11-09 15:41:16 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 15:41:36 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 15:41:56 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 15:42:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:42:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:42:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:42:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:42:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:42:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:42:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:42:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:42:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:42:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:42:02 [DEBUG   ] Finished updating board flags in 2.013 seconds
2015-11-09 15:42:16 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 15:42:36 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 15:42:48 [DEBUG   ] Got command RPT from MCS: ref #528761
2015-11-09 15:42:48 [DEBUG   ] summary = NORMAL
2015-11-09 15:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528761  14 57335 56568575 A NORMALNORMAL'
2015-11-09 15:42:48 [DEBUG   ] Got command RPT from MCS: ref #528762
2015-11-09 15:42:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 15:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528762  48 57335 56568575 A NORMALSystem calibrated and operating normally'
2015-11-09 15:42:57 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 15:43:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:43:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:43:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:43:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:43:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:43:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:43:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:43:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:43:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:43:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:43:02 [DEBUG   ] Finished updating board flags in 1.991 seconds
2015-11-09 15:43:17 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 15:43:37 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 15:43:57 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-09 15:44:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:44:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:44:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:44:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:44:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:44:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:44:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:44:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:44:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:44:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:44:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 15:44:02 [DEBUG   ] Finished updating board flags in 1.816 seconds
2015-11-09 15:44:17 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 15:44:37 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 15:44:57 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 15:45:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:45:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:45:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:45:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:45:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:45:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:45:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:45:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:45:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:45:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:45:02 [DEBUG   ] Finished updating board flags in 1.785 seconds
2015-11-09 15:45:09 [DEBUG   ] Got command RPT from MCS: ref #528782
2015-11-09 15:45:09 [DEBUG   ] summary = NORMAL
2015-11-09 15:45:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528782  14 57335 56709635 A NORMALNORMAL'
2015-11-09 15:45:17 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 15:45:37 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-09 15:45:57 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 15:46:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:46:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:46:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:46:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:46:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:46:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:46:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:46:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:46:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:46:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:46:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 15:46:02 [DEBUG   ] Finished updating board flags in 1.611 seconds
2015-11-09 15:46:17 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 15:46:37 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 15:46:57 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 15:47:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:47:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:47:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:47:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:47:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:47:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:47:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:47:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:47:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:47:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:47:02 [DEBUG   ] Finished updating board flags in 1.611 seconds
2015-11-09 15:47:17 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 15:47:37 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 15:47:48 [DEBUG   ] Got command RPT from MCS: ref #528791
2015-11-09 15:47:48 [DEBUG   ] summary = NORMAL
2015-11-09 15:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528791  14 57335 56868567 A NORMALNORMAL'
2015-11-09 15:47:48 [DEBUG   ] Got command RPT from MCS: ref #528792
2015-11-09 15:47:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 15:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528792  48 57335 56868567 A NORMALSystem calibrated and operating normally'
2015-11-09 15:47:58 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 15:48:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:48:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:48:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:48:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:48:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:48:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:48:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:48:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:48:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:48:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:48:02 [DEBUG   ] Finished updating board flags in 1.588 seconds
2015-11-09 15:48:18 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 15:48:38 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 15:48:58 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 15:49:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:49:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:49:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:49:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:49:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:49:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:49:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:49:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:49:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:49:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:49:02 [DEBUG   ] Finished updating board flags in 1.416 seconds
2015-11-09 15:49:18 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 15:49:38 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 15:49:58 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 15:50:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:50:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:50:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:50:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:50:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:50:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:50:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:50:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:50:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:50:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:50:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 15:50:02 [DEBUG   ] Finished updating board flags in 1.388 seconds
2015-11-09 15:50:09 [DEBUG   ] Got command RPT from MCS: ref #528808
2015-11-09 15:50:09 [DEBUG   ] summary = NORMAL
2015-11-09 15:50:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528808  14 57335 57009739 A NORMALNORMAL'
2015-11-09 15:50:18 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 15:50:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 15:50:58 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 15:51:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:51:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:51:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:51:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:51:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:51:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:51:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:51:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:51:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:51:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:51:02 [DEBUG   ] Finished updating board flags in 1.514 seconds
2015-11-09 15:51:18 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2015-11-09 15:51:38 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 15:51:58 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 15:52:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:52:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:52:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:52:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:52:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:52:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:52:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:52:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:52:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:52:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:52:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 15:52:03 [DEBUG   ] Finished updating board flags in 2.196 seconds
2015-11-09 15:52:18 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 15:52:38 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 15:52:48 [DEBUG   ] Got command RPT from MCS: ref #528812
2015-11-09 15:52:48 [DEBUG   ] summary = NORMAL
2015-11-09 15:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528812  14 57335 57168567 A NORMALNORMAL'
2015-11-09 15:52:48 [DEBUG   ] Got command RPT from MCS: ref #528813
2015-11-09 15:52:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-09 15:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528813  48 57335 57168567 A NORMALSystem calibrated and operating normally'
2015-11-09 15:52:58 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 15:53:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:53:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:53:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:53:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:53:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:53:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:53:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:53:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:53:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:53:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:53:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2015-11-09 15:53:03 [CRITICAL] MGT underflow detected on beam(s) 1
2015-11-09 15:53:03 [DEBUG   ] Finished updating board flags in 1.928 seconds
2015-11-09 15:53:18 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 15:53:38 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 15:53:58 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 15:54:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:54:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:54:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:54:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:54:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:54:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:54:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:54:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:54:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:54:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:54:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 15:54:03 [DEBUG   ] Finished updating board flags in 1.778 seconds
2015-11-09 15:54:19 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 15:54:39 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 15:54:59 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 15:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:55:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:55:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:55:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:55:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:55:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:55:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 15:55:03 [DEBUG   ] Finished updating board flags in 1.614 seconds
2015-11-09 15:55:09 [DEBUG   ] Got command RPT from MCS: ref #528833
2015-11-09 15:55:09 [DEBUG   ] summary = ERROR
2015-11-09 15:55:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528833  13 57335 57309839 A  ERRORERROR'
2015-11-09 15:55:10 [DEBUG   ] Got command RPT from MCS: ref #528834
2015-11-09 15:55:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 15:55:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528834  77 57335 57310031 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 15:55:10 [DEBUG   ] Got command RPT from MCS: ref #528835
2015-11-09 15:55:10 [DEBUG   ] summary = ERROR
2015-11-09 15:55:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528835  13 57335 57310235 A  ERRORERROR'
2015-11-09 15:55:10 [DEBUG   ] Got command RPT from MCS: ref #528836
2015-11-09 15:55:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 15:55:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528836  77 57335 57310439 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 15:55:10 [DEBUG   ] Got command RPT from MCS: ref #528837
2015-11-09 15:55:10 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 15:55:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528837  12 57335 57310643 A  ERRORB�ÌÍ'
2015-11-09 15:55:19 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 15:55:39 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 15:55:59 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 15:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:56:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:56:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:56:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:56:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:56:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:56:03 [DEBUG   ] Finished updating board flags in 1.577 seconds
2015-11-09 15:56:19 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 15:56:39 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 15:56:59 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 15:57:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:57:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:57:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:57:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:57:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:57:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:57:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:57:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:57:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:57:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:57:03 [DEBUG   ] Finished updating board flags in 1.413 seconds
2015-11-09 15:57:19 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 15:57:39 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 15:57:48 [DEBUG   ] Got command RPT from MCS: ref #528846
2015-11-09 15:57:48 [DEBUG   ] summary = ERROR
2015-11-09 15:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528846  13 57335 57468575 A  ERRORERROR'
2015-11-09 15:57:48 [DEBUG   ] Got command RPT from MCS: ref #528847
2015-11-09 15:57:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 15:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528847  77 57335 57468575 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 15:57:59 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 15:58:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:58:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:58:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:58:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:58:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:58:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:58:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:58:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:58:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:58:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:58:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 15:58:03 [DEBUG   ] Finished updating board flags in 1.418 seconds
2015-11-09 15:58:19 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 15:58:39 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 15:58:59 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 15:59:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 15:59:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 15:59:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 15:59:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 15:59:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 15:59:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 15:59:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 15:59:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 15:59:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 15:59:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 15:59:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 15:59:03 [DEBUG   ] Finished updating board flags in 1.414 seconds
2015-11-09 15:59:20 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 15:59:40 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 16:00:00 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 16:00:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:00:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:00:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:00:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:00:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:00:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 16:00:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:00:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:00:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:00:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:00:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:00:03 [DEBUG   ] Finished updating board flags in 1.412 seconds
2015-11-09 16:00:09 [DEBUG   ] Got command RPT from MCS: ref #528866
2015-11-09 16:00:09 [DEBUG   ] summary = ERROR
2015-11-09 16:00:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528866  13 57335 57609923 A  ERRORERROR'
2015-11-09 16:00:10 [DEBUG   ] Got command RPT from MCS: ref #528867
2015-11-09 16:00:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:00:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528867  77 57335 57610127 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:00:10 [DEBUG   ] Got command RPT from MCS: ref #528868
2015-11-09 16:00:10 [DEBUG   ] summary = ERROR
2015-11-09 16:00:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528868  13 57335 57610331 A  ERRORERROR'
2015-11-09 16:00:10 [DEBUG   ] Got command RPT from MCS: ref #528869
2015-11-09 16:00:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:00:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528869  77 57335 57610535 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:00:10 [DEBUG   ] Got command RPT from MCS: ref #528870
2015-11-09 16:00:10 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 16:00:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528870  12 57335 57610727 A  ERRORBŒÌÍ'
2015-11-09 16:00:20 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 16:00:40 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 16:01:00 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 16:01:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:01:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:01:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:01:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:01:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:01:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:01:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:01:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:01:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:01:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:01:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2015-11-09 16:01:03 [CRITICAL] MGT underflow detected on beam(s) 1
2015-11-09 16:01:03 [DEBUG   ] Finished updating board flags in 1.413 seconds
2015-11-09 16:01:20 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 16:01:40 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 16:02:00 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 16:02:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:02:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:02:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:02:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:02:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:02:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:02:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:02:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:02:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:02:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:02:03 [DEBUG   ] Finished updating board flags in 1.415 seconds
2015-11-09 16:02:20 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 16:02:40 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 16:02:48 [DEBUG   ] Got command RPT from MCS: ref #528874
2015-11-09 16:02:48 [DEBUG   ] summary = ERROR
2015-11-09 16:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528874  13 57335 57768579 A  ERRORERROR'
2015-11-09 16:02:48 [DEBUG   ] Got command RPT from MCS: ref #528875
2015-11-09 16:02:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528875  77 57335 57768579 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:03:00 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 16:03:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:03:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:03:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:03:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:03:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:03:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:03:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:03:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:03:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:03:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:03:03 [DEBUG   ] Finished updating board flags in 1.415 seconds
2015-11-09 16:03:20 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-09 16:03:40 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 16:04:01 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-09 16:04:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:04:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:04:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:04:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:04:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:04:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:04:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:04:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:04:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:04:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:04:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 16:04:03 [DEBUG   ] Finished updating board flags in 1.418 seconds
2015-11-09 16:04:21 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 16:04:41 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 16:05:01 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 16:05:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:05:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:05:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:05:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:05:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:05:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:05:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:05:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:05:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:05:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:05:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 16:05:03 [DEBUG   ] Finished updating board flags in 1.414 seconds
2015-11-09 16:05:10 [DEBUG   ] Got command RPT from MCS: ref #528892
2015-11-09 16:05:10 [DEBUG   ] summary = ERROR
2015-11-09 16:05:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528892  13 57335 57910011 A  ERRORERROR'
2015-11-09 16:05:10 [DEBUG   ] Got command RPT from MCS: ref #528893
2015-11-09 16:05:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:05:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528893  77 57335 57910215 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:05:10 [DEBUG   ] Got command RPT from MCS: ref #528894
2015-11-09 16:05:10 [DEBUG   ] summary = ERROR
2015-11-09 16:05:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528894  13 57335 57910419 A  ERRORERROR'
2015-11-09 16:05:10 [DEBUG   ] Got command RPT from MCS: ref #528895
2015-11-09 16:05:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:05:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528895  77 57335 57910623 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:05:10 [DEBUG   ] Got command RPT from MCS: ref #528896
2015-11-09 16:05:10 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 16:05:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528896  12 57335 57910815 A  ERRORB�ÌÍ'
2015-11-09 16:05:21 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 16:05:41 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 16:06:01 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 16:06:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:06:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:06:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:06:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:06:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:06:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:06:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:06:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:06:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:06:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:06:03 [DEBUG   ] Finished updating board flags in 1.413 seconds
2015-11-09 16:06:21 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 16:06:41 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 16:07:01 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 16:07:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:07:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:07:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:07:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:07:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:07:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:07:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:07:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:07:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:07:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:07:03 [DEBUG   ] Finished updating board flags in 1.429 seconds
2015-11-09 16:07:21 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 16:07:41 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 16:07:48 [DEBUG   ] Got command RPT from MCS: ref #528905
2015-11-09 16:07:48 [DEBUG   ] summary = ERROR
2015-11-09 16:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528905  13 57335 58068579 A  ERRORERROR'
2015-11-09 16:07:48 [DEBUG   ] Got command RPT from MCS: ref #528906
2015-11-09 16:07:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528906  77 57335 58068579 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:08:01 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 16:08:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:08:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:08:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:08:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:08:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:08:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:08:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:08:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:08:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:08:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:08:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 16:08:03 [DEBUG   ] Finished updating board flags in 1.429 seconds
2015-11-09 16:08:21 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 16:08:41 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 16:09:01 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 16:09:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:09:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:09:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:09:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:09:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:09:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:09:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:09:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:09:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:09:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:09:03 [DEBUG   ] Finished updating board flags in 1.433 seconds
2015-11-09 16:09:21 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 16:09:41 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 16:10:02 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 16:10:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:10:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:10:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:10:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:10:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:10:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:10:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:10:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:10:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:10:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:10:04 [DEBUG   ] Finished updating board flags in 2.216 seconds
2015-11-09 16:10:10 [DEBUG   ] Got command RPT from MCS: ref #528925
2015-11-09 16:10:10 [DEBUG   ] summary = ERROR
2015-11-09 16:10:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528925  13 57335 58210103 A  ERRORERROR'
2015-11-09 16:10:10 [DEBUG   ] Got command RPT from MCS: ref #528926
2015-11-09 16:10:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:10:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528926  77 57335 58210295 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:10:10 [DEBUG   ] Got command RPT from MCS: ref #528927
2015-11-09 16:10:10 [DEBUG   ] summary = ERROR
2015-11-09 16:10:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528927  13 57335 58210499 A  ERRORERROR'
2015-11-09 16:10:10 [DEBUG   ] Got command RPT from MCS: ref #528928
2015-11-09 16:10:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:10:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528928  77 57335 58210703 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:10:10 [DEBUG   ] Got command RPT from MCS: ref #528929
2015-11-09 16:10:10 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 16:10:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528929  12 57335 58210899 A  ERRORBŒÌÍ'
2015-11-09 16:10:22 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-09 16:10:42 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 16:11:02 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 16:11:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:11:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:11:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:11:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:11:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:11:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:11:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:11:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:11:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:11:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:11:04 [DEBUG   ] Finished updating board flags in 2.175 seconds
2015-11-09 16:11:22 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 16:11:42 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 16:12:02 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 16:12:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:12:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:12:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:12:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:12:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:12:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:12:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:12:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:12:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:12:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:12:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 16:12:04 [DEBUG   ] Finished updating board flags in 2.035 seconds
2015-11-09 16:12:22 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 16:12:42 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 16:12:48 [DEBUG   ] Got command RPT from MCS: ref #528933
2015-11-09 16:12:48 [DEBUG   ] summary = ERROR
2015-11-09 16:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528933  13 57335 58368579 A  ERRORERROR'
2015-11-09 16:12:48 [DEBUG   ] Got command RPT from MCS: ref #528934
2015-11-09 16:12:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528934  77 57335 58368579 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:13:02 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 16:13:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:13:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:13:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:13:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:13:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:13:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:13:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:13:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:13:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:13:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:13:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 16:13:04 [DEBUG   ] Finished updating board flags in 1.972 seconds
2015-11-09 16:13:22 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 16:13:42 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 16:14:02 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 16:14:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:14:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:14:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:14:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:14:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:14:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:14:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:14:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:14:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:14:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:14:04 [DEBUG   ] Finished updating board flags in 1.816 seconds
2015-11-09 16:14:22 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 16:14:42 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 16:15:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:15:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:15:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:15:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:15:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:15:03 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 16:15:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:15:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:15:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:15:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:15:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:15:04 [DEBUG   ] Finished updating board flags in 1.787 seconds
2015-11-09 16:15:10 [DEBUG   ] Got command RPT from MCS: ref #528954
2015-11-09 16:15:10 [DEBUG   ] summary = ERROR
2015-11-09 16:15:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528954  13 57335 58510183 A  ERRORERROR'
2015-11-09 16:15:10 [DEBUG   ] Got command RPT from MCS: ref #528955
2015-11-09 16:15:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:15:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528955  77 57335 58510379 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:15:10 [DEBUG   ] Got command RPT from MCS: ref #528956
2015-11-09 16:15:10 [DEBUG   ] summary = ERROR
2015-11-09 16:15:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528956  13 57335 58510583 A  ERRORERROR'
2015-11-09 16:15:10 [DEBUG   ] Got command RPT from MCS: ref #528957
2015-11-09 16:15:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:15:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528957  77 57335 58510787 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:15:10 [DEBUG   ] Got command RPT from MCS: ref #528958
2015-11-09 16:15:10 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 16:15:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528958  12 57335 58510991 A  ERRORB�ÌÍ'
2015-11-09 16:15:23 [DEBUG   ] Finished updating ant stats in 1.210 seconds
2015-11-09 16:15:43 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 16:16:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:16:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:16:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:16:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:16:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:16:03 [DEBUG   ] Finished updating ant stats in 1.250 seconds
2015-11-09 16:16:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:16:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:16:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:16:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:16:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:16:04 [DEBUG   ] Finished updating board flags in 1.619 seconds
2015-11-09 16:16:23 [DEBUG   ] Finished updating ant stats in 1.154 seconds
2015-11-09 16:16:43 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 16:17:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:17:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:17:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:17:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:17:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:17:03 [DEBUG   ] Finished updating ant stats in 1.051 seconds
2015-11-09 16:17:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:17:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:17:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:17:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:17:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:17:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 16:17:04 [DEBUG   ] Finished updating board flags in 1.610 seconds
2015-11-09 16:17:23 [DEBUG   ] Finished updating ant stats in 0.951 seconds
2015-11-09 16:17:43 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 16:17:48 [DEBUG   ] Got command RPT from MCS: ref #528967
2015-11-09 16:17:48 [DEBUG   ] summary = ERROR
2015-11-09 16:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528967  13 57335 58668579 A  ERRORERROR'
2015-11-09 16:17:48 [DEBUG   ] Got command RPT from MCS: ref #528968
2015-11-09 16:17:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528968  77 57335 58668579 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:18:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:18:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:18:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:18:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:18:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:18:03 [DEBUG   ] Finished updating ant stats in 0.852 seconds
2015-11-09 16:18:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:18:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:18:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:18:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:18:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:18:04 [DEBUG   ] Finished updating board flags in 1.583 seconds
2015-11-09 16:18:23 [DEBUG   ] Finished updating ant stats in 0.752 seconds
2015-11-09 16:18:43 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 16:19:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:19:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:19:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:19:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:19:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:19:03 [DEBUG   ] Finished updating ant stats in 0.654 seconds
2015-11-09 16:19:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:19:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:19:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:19:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:19:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:19:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 16:19:04 [DEBUG   ] Finished updating board flags in 1.416 seconds
2015-11-09 16:19:23 [DEBUG   ] Finished updating ant stats in 0.553 seconds
2015-11-09 16:19:44 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2015-11-09 16:20:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:20:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:20:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:20:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:20:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:20:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:20:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:20:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:20:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:20:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:20:04 [DEBUG   ] Finished updating ant stats in 1.251 seconds
2015-11-09 16:20:04 [DEBUG   ] Finished updating board flags in 1.460 seconds
2015-11-09 16:20:10 [DEBUG   ] Got command RPT from MCS: ref #528984
2015-11-09 16:20:10 [DEBUG   ] summary = ERROR
2015-11-09 16:20:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528984  13 57335 58810271 A  ERRORERROR'
2015-11-09 16:20:10 [DEBUG   ] Got command RPT from MCS: ref #528985
2015-11-09 16:20:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:20:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528985  77 57335 58810467 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:20:10 [DEBUG   ] Got command RPT from MCS: ref #528986
2015-11-09 16:20:10 [DEBUG   ] summary = ERROR
2015-11-09 16:20:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528986  13 57335 58810671 A  ERRORERROR'
2015-11-09 16:20:10 [DEBUG   ] Got command RPT from MCS: ref #528987
2015-11-09 16:20:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:20:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528987  77 57335 58810867 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:20:11 [DEBUG   ] Got command RPT from MCS: ref #528988
2015-11-09 16:20:11 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 16:20:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528988  12 57335 58811071 A  ERRORB‹ÌÍ'
2015-11-09 16:20:24 [DEBUG   ] Finished updating ant stats in 1.161 seconds
2015-11-09 16:20:44 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 16:21:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999411
2015-11-09 16:21:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999411
2015-11-09 16:21:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999411
2015-11-09 16:21:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999411
2015-11-09 16:21:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999411
2015-11-09 16:21:04 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 16:21:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999411
2015-11-09 16:21:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999411
2015-11-09 16:21:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999411
2015-11-09 16:21:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999411
2015-11-09 16:21:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999411
2015-11-09 16:21:04 [DEBUG   ] Finished updating ant stats in 0.844 seconds
2015-11-09 16:21:04 [DEBUG   ] Finished updating board flags in 1.487 seconds
2015-11-09 16:21:24 [DEBUG   ] Finished updating ant stats in 0.757 seconds
2015-11-09 16:21:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999215
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999215
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999215
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999215
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999215
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 196000019
2015-11-09 16:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999215
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999215
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999215
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999215
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999215
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 196000019
2015-11-09 16:22:04 [DEBUG   ] Finished updating ant stats in 0.644 seconds
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 196000019
2015-11-09 16:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 196000019
2015-11-09 16:22:04 [DEBUG   ] Finished updating board flags in 1.503 seconds
2015-11-09 16:22:24 [DEBUG   ] Finished updating ant stats in 0.559 seconds
2015-11-09 16:22:45 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 16:22:48 [DEBUG   ] Got command RPT from MCS: ref #528992
2015-11-09 16:22:48 [DEBUG   ] summary = ERROR
2015-11-09 16:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528992  13 57335 58968583 A  ERRORERROR'
2015-11-09 16:22:48 [DEBUG   ] Got command RPT from MCS: ref #528993
2015-11-09 16:22:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   528993  77 57335 58968583 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999215
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999215
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999215
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999215
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999215
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 16:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999215
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999215
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999215
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999215
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999215
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 16:23:05 [DEBUG   ] Finished updating ant stats in 1.247 seconds
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 16:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 16:23:05 [DEBUG   ] Finished updating board flags in 2.237 seconds
2015-11-09 16:23:25 [DEBUG   ] Finished updating ant stats in 1.162 seconds
2015-11-09 16:23:45 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 16:24:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999215
2015-11-09 16:24:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999215
2015-11-09 16:24:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999215
2015-11-09 16:24:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999215
2015-11-09 16:24:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999215
2015-11-09 16:24:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999215
2015-11-09 16:24:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999215
2015-11-09 16:24:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999215
2015-11-09 16:24:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999215
2015-11-09 16:24:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999215
2015-11-09 16:24:05 [DEBUG   ] Finished updating ant stats in 1.041 seconds
2015-11-09 16:24:05 [DEBUG   ] Finished updating board flags in 2.176 seconds
2015-11-09 16:24:25 [DEBUG   ] Finished updating ant stats in 0.959 seconds
2015-11-09 16:24:45 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 16:25:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999215
2015-11-09 16:25:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999215
2015-11-09 16:25:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999215
2015-11-09 16:25:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999215
2015-11-09 16:25:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999215
2015-11-09 16:25:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 16:25:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999215
2015-11-09 16:25:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999215
2015-11-09 16:25:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999215
2015-11-09 16:25:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999215
2015-11-09 16:25:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999215
2015-11-09 16:25:05 [DEBUG   ] Finished updating ant stats in 0.845 seconds
2015-11-09 16:25:05 [DEBUG   ] Finished updating board flags in 2.007 seconds
2015-11-09 16:25:10 [DEBUG   ] Got command RPT from MCS: ref #529013
2015-11-09 16:25:10 [DEBUG   ] summary = ERROR
2015-11-09 16:25:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529013  13 57335 59110351 A  ERRORERROR'
2015-11-09 16:25:10 [DEBUG   ] Got command RPT from MCS: ref #529014
2015-11-09 16:25:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:25:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529014  77 57335 59110555 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:25:10 [DEBUG   ] Got command RPT from MCS: ref #529015
2015-11-09 16:25:10 [DEBUG   ] summary = ERROR
2015-11-09 16:25:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529015  13 57335 59110747 A  ERRORERROR'
2015-11-09 16:25:10 [DEBUG   ] Got command RPT from MCS: ref #529016
2015-11-09 16:25:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:25:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529016  77 57335 59110951 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:25:11 [DEBUG   ] Got command RPT from MCS: ref #529017
2015-11-09 16:25:11 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 16:25:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529017  12 57335 59111155 A  ERRORBŒÌÍ'
2015-11-09 16:25:25 [DEBUG   ] Finished updating ant stats in 0.756 seconds
2015-11-09 16:25:45 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 16:26:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999215
2015-11-09 16:26:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999215
2015-11-09 16:26:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999215
2015-11-09 16:26:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999215
2015-11-09 16:26:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999215
2015-11-09 16:26:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999215
2015-11-09 16:26:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999215
2015-11-09 16:26:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999215
2015-11-09 16:26:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999215
2015-11-09 16:26:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999215
2015-11-09 16:26:05 [DEBUG   ] Finished updating ant stats in 0.646 seconds
2015-11-09 16:26:05 [DEBUG   ] Finished updating board flags in 1.815 seconds
2015-11-09 16:26:25 [DEBUG   ] Finished updating ant stats in 0.561 seconds
2015-11-09 16:26:46 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999117
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999117
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999117
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999117
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999117
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-09 16:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999117
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999117
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999117
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999117
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999117
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-09 16:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-09 16:27:05 [DEBUG   ] Finished updating board flags in 1.804 seconds
2015-11-09 16:27:06 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 16:27:26 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 16:27:46 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 16:27:48 [DEBUG   ] Got command RPT from MCS: ref #529026
2015-11-09 16:27:48 [DEBUG   ] summary = ERROR
2015-11-09 16:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529026  13 57335 59268575 A  ERRORERROR'
2015-11-09 16:27:48 [DEBUG   ] Got command RPT from MCS: ref #529027
2015-11-09 16:27:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529027  77 57335 59268575 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999117
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999117
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999117
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999117
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999117
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 16:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999117
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999117
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999117
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999117
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999117
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 16:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 16:28:05 [DEBUG   ] Finished updating board flags in 1.795 seconds
2015-11-09 16:28:06 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 16:28:26 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 16:28:46 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 16:29:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999117
2015-11-09 16:29:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999117
2015-11-09 16:29:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999117
2015-11-09 16:29:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999117
2015-11-09 16:29:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999117
2015-11-09 16:29:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999117
2015-11-09 16:29:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999117
2015-11-09 16:29:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999117
2015-11-09 16:29:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999117
2015-11-09 16:29:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999117
2015-11-09 16:29:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 16:29:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 16:29:05 [DEBUG   ] Finished updating board flags in 1.614 seconds
2015-11-09 16:29:06 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 16:29:26 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 16:29:46 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 16:30:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999117
2015-11-09 16:30:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999117
2015-11-09 16:30:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999117
2015-11-09 16:30:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999117
2015-11-09 16:30:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999117
2015-11-09 16:30:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999117
2015-11-09 16:30:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999117
2015-11-09 16:30:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999117
2015-11-09 16:30:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999117
2015-11-09 16:30:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999117
2015-11-09 16:30:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 16:30:05 [DEBUG   ] Finished updating board flags in 1.537 seconds
2015-11-09 16:30:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 16:30:10 [DEBUG   ] Got command RPT from MCS: ref #529046
2015-11-09 16:30:10 [DEBUG   ] summary = ERROR
2015-11-09 16:30:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529046  13 57335 59410435 A  ERRORERROR'
2015-11-09 16:30:10 [DEBUG   ] Got command RPT from MCS: ref #529047
2015-11-09 16:30:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:30:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529047  77 57335 59410627 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:30:10 [DEBUG   ] Got command RPT from MCS: ref #529048
2015-11-09 16:30:10 [DEBUG   ] summary = ERROR
2015-11-09 16:30:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529048  13 57335 59410831 A  ERRORERROR'
2015-11-09 16:30:11 [DEBUG   ] Got command RPT from MCS: ref #529049
2015-11-09 16:30:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:30:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529049  77 57335 59411045 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:30:11 [DEBUG   ] Got command RPT from MCS: ref #529050
2015-11-09 16:30:11 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 16:30:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529050  12 57335 59411235 A  ERRORBŒÌÍ'
2015-11-09 16:30:26 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 16:30:46 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 16:31:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999117
2015-11-09 16:31:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999117
2015-11-09 16:31:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999117
2015-11-09 16:31:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999117
2015-11-09 16:31:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999117
2015-11-09 16:31:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999117
2015-11-09 16:31:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999117
2015-11-09 16:31:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999117
2015-11-09 16:31:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999117
2015-11-09 16:31:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999117
2015-11-09 16:31:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 16:31:05 [DEBUG   ] Finished updating board flags in 1.438 seconds
2015-11-09 16:31:07 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 16:31:27 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 16:31:47 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 16:32:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999117
2015-11-09 16:32:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999117
2015-11-09 16:32:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999117
2015-11-09 16:32:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999117
2015-11-09 16:32:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999117
2015-11-09 16:32:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999117
2015-11-09 16:32:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999117
2015-11-09 16:32:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999117
2015-11-09 16:32:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999117
2015-11-09 16:32:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999117
2015-11-09 16:32:05 [DEBUG   ] Finished updating board flags in 1.364 seconds
2015-11-09 16:32:07 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 16:32:27 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 16:32:47 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 16:32:48 [DEBUG   ] Got command RPT from MCS: ref #529054
2015-11-09 16:32:48 [DEBUG   ] summary = ERROR
2015-11-09 16:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529054  13 57335 59568579 A  ERRORERROR'
2015-11-09 16:32:48 [DEBUG   ] Got command RPT from MCS: ref #529055
2015-11-09 16:32:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529055  77 57335 59568579 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:33:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999117
2015-11-09 16:33:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999117
2015-11-09 16:33:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999117
2015-11-09 16:33:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999117
2015-11-09 16:33:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999117
2015-11-09 16:33:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999117
2015-11-09 16:33:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999117
2015-11-09 16:33:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999117
2015-11-09 16:33:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999117
2015-11-09 16:33:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999117
2015-11-09 16:33:05 [DEBUG   ] Finished updating board flags in 1.378 seconds
2015-11-09 16:33:07 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-09 16:33:27 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 16:33:47 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 16:34:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999117
2015-11-09 16:34:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999117
2015-11-09 16:34:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999117
2015-11-09 16:34:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999117
2015-11-09 16:34:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999117
2015-11-09 16:34:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999117
2015-11-09 16:34:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999117
2015-11-09 16:34:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999117
2015-11-09 16:34:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999117
2015-11-09 16:34:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999117
2015-11-09 16:34:06 [DEBUG   ] Finished updating board flags in 2.083 seconds
2015-11-09 16:34:07 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 16:34:27 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 16:34:47 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 16:35:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999117
2015-11-09 16:35:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999117
2015-11-09 16:35:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999117
2015-11-09 16:35:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999117
2015-11-09 16:35:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999117
2015-11-09 16:35:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999117
2015-11-09 16:35:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999117
2015-11-09 16:35:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999117
2015-11-09 16:35:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999117
2015-11-09 16:35:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999117
2015-11-09 16:35:06 [DEBUG   ] Finished updating board flags in 1.980 seconds
2015-11-09 16:35:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 16:35:10 [DEBUG   ] Got command RPT from MCS: ref #529072
2015-11-09 16:35:10 [DEBUG   ] summary = ERROR
2015-11-09 16:35:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529072  13 57335 59710511 A  ERRORERROR'
2015-11-09 16:35:10 [DEBUG   ] Got command RPT from MCS: ref #529073
2015-11-09 16:35:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:35:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529073  77 57335 59710715 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:35:10 [DEBUG   ] Got command RPT from MCS: ref #529074
2015-11-09 16:35:10 [DEBUG   ] summary = ERROR
2015-11-09 16:35:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529074  13 57335 59710919 A  ERRORERROR'
2015-11-09 16:35:11 [DEBUG   ] Got command RPT from MCS: ref #529075
2015-11-09 16:35:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:35:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529075  77 57335 59711123 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:35:11 [DEBUG   ] Got command RPT from MCS: ref #529076
2015-11-09 16:35:11 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 16:35:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529076  12 57335 59711315 A  ERRORB‹ÌÍ'
2015-11-09 16:35:28 [DEBUG   ] Finished updating ant stats in 1.799 seconds
2015-11-09 16:35:47 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 16:36:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999117
2015-11-09 16:36:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999117
2015-11-09 16:36:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999117
2015-11-09 16:36:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999117
2015-11-09 16:36:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999117
2015-11-09 16:36:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999117
2015-11-09 16:36:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999117
2015-11-09 16:36:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999117
2015-11-09 16:36:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999117
2015-11-09 16:36:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999117
2015-11-09 16:36:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 16:36:06 [DEBUG   ] Finished updating board flags in 1.819 seconds
2015-11-09 16:36:07 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 16:36:27 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 16:36:48 [DEBUG   ] Finished updating ant stats in 1.599 seconds
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-09 16:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-09 16:37:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 16:37:06 [DEBUG   ] Finished updating board flags in 1.769 seconds
2015-11-09 16:37:08 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 16:37:28 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 16:37:48 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-09 16:37:48 [DEBUG   ] Got command RPT from MCS: ref #529085
2015-11-09 16:37:48 [DEBUG   ] summary = ERROR
2015-11-09 16:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529085  13 57335 59868579 A  ERRORERROR'
2015-11-09 16:37:48 [DEBUG   ] Got command RPT from MCS: ref #529086
2015-11-09 16:37:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529086  77 57335 59868579 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 16:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 16:38:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 16:38:06 [DEBUG   ] Finished updating board flags in 1.608 seconds
2015-11-09 16:38:08 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 16:38:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 16:38:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 16:39:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:39:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:39:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:39:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:39:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:39:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:39:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:39:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:39:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:39:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:39:06 [DEBUG   ] Finished updating board flags in 1.611 seconds
2015-11-09 16:39:08 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 16:39:28 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 16:39:48 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 16:40:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:40:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:40:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:40:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:40:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:40:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:40:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:40:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:40:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:40:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:40:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 16:40:06 [DEBUG   ] Finished updating board flags in 1.573 seconds
2015-11-09 16:40:08 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 16:40:10 [DEBUG   ] Got command RPT from MCS: ref #529105
2015-11-09 16:40:10 [DEBUG   ] summary = ERROR
2015-11-09 16:40:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529105  13 57335 60010607 A  ERRORERROR'
2015-11-09 16:40:10 [DEBUG   ] Got command RPT from MCS: ref #529106
2015-11-09 16:40:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:40:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529106  77 57335 60010799 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:40:11 [DEBUG   ] Got command RPT from MCS: ref #529107
2015-11-09 16:40:11 [DEBUG   ] summary = ERROR
2015-11-09 16:40:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529107  13 57335 60011003 A  ERRORERROR'
2015-11-09 16:40:11 [DEBUG   ] Got command RPT from MCS: ref #529108
2015-11-09 16:40:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:40:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529108  77 57335 60011207 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:40:11 [DEBUG   ] Got command RPT from MCS: ref #529109
2015-11-09 16:40:11 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 16:40:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529109  12 57335 60011399 A  ERRORB�ÌÍ'
2015-11-09 16:40:28 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 16:40:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 16:41:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:41:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:41:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:41:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:41:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:41:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:41:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:41:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:41:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:41:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:41:06 [DEBUG   ] Finished updating board flags in 1.413 seconds
2015-11-09 16:41:08 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 16:41:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 16:41:48 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 16:42:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:42:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:42:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:42:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:42:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:42:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:42:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:42:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:42:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:42:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:42:06 [DEBUG   ] Finished updating board flags in 1.541 seconds
2015-11-09 16:42:08 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 16:42:28 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 16:42:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 16:42:48 [DEBUG   ] Got command RPT from MCS: ref #529113
2015-11-09 16:42:48 [DEBUG   ] summary = ERROR
2015-11-09 16:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529113  13 57335 60168579 A  ERRORERROR'
2015-11-09 16:42:48 [DEBUG   ] Got command RPT from MCS: ref #529114
2015-11-09 16:42:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529114  77 57335 60168579 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:43:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:43:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:43:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:43:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:43:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:43:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:43:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:43:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:43:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:43:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:43:06 [DEBUG   ] Finished updating board flags in 1.530 seconds
2015-11-09 16:43:08 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 16:43:29 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 16:43:49 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 16:44:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:44:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:44:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:44:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:44:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:44:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:44:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:44:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:44:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:44:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:44:07 [DEBUG   ] Finished updating board flags in 2.212 seconds
2015-11-09 16:44:09 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 16:44:29 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 16:44:49 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 16:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:45:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:45:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:45:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:45:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:45:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:45:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 16:45:07 [DEBUG   ] Finished updating board flags in 2.120 seconds
2015-11-09 16:45:09 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 16:45:10 [DEBUG   ] Got command RPT from MCS: ref #529134
2015-11-09 16:45:10 [DEBUG   ] summary = ERROR
2015-11-09 16:45:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529134  13 57335 60310683 A  ERRORERROR'
2015-11-09 16:45:10 [DEBUG   ] Got command RPT from MCS: ref #529135
2015-11-09 16:45:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:45:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529135  77 57335 60310887 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:45:11 [DEBUG   ] Got command RPT from MCS: ref #529136
2015-11-09 16:45:11 [DEBUG   ] summary = ERROR
2015-11-09 16:45:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529136  13 57335 60311091 A  ERRORERROR'
2015-11-09 16:45:11 [DEBUG   ] Got command RPT from MCS: ref #529137
2015-11-09 16:45:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:45:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529137  77 57335 60311283 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:45:11 [DEBUG   ] Got command RPT from MCS: ref #529138
2015-11-09 16:45:11 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 16:45:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529138  12 57335 60311487 A  ERRORBŒÌÍ'
2015-11-09 16:45:29 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 16:45:49 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 16:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:46:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:46:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:46:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:46:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:46:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:46:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 16:46:07 [DEBUG   ] Finished updating board flags in 2.010 seconds
2015-11-09 16:46:09 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 16:46:29 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 16:46:49 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 16:47:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:47:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:47:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:47:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:47:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:47:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:47:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:47:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:47:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:47:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:47:07 [DEBUG   ] Finished updating board flags in 1.994 seconds
2015-11-09 16:47:09 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 16:47:29 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 16:47:48 [DEBUG   ] Got command RPT from MCS: ref #529147
2015-11-09 16:47:48 [DEBUG   ] summary = ERROR
2015-11-09 16:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529147  13 57335 60468587 A  ERRORERROR'
2015-11-09 16:47:48 [DEBUG   ] Got command RPT from MCS: ref #529148
2015-11-09 16:47:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529148  77 57335 60468588 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:47:49 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 16:48:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:48:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:48:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:48:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:48:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:48:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:48:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:48:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:48:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:48:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:48:07 [DEBUG   ] Finished updating board flags in 1.812 seconds
2015-11-09 16:48:09 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 16:48:29 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 16:48:49 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 16:49:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:49:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:49:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:49:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:49:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:49:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:49:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:49:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:49:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:49:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:49:07 [DEBUG   ] Finished updating board flags in 1.781 seconds
2015-11-09 16:49:09 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 16:49:30 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 16:49:50 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-09 16:50:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:50:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:50:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:50:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:50:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:50:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:50:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:50:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:50:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:50:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:50:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 16:50:07 [DEBUG   ] Finished updating board flags in 1.614 seconds
2015-11-09 16:50:10 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 16:50:10 [DEBUG   ] Got command RPT from MCS: ref #529164
2015-11-09 16:50:10 [DEBUG   ] summary = ERROR
2015-11-09 16:50:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529164  13 57335 60610779 A  ERRORERROR'
2015-11-09 16:50:10 [DEBUG   ] Got command RPT from MCS: ref #529165
2015-11-09 16:50:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:50:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529165  77 57335 60610983 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:50:11 [DEBUG   ] Got command RPT from MCS: ref #529166
2015-11-09 16:50:11 [DEBUG   ] summary = ERROR
2015-11-09 16:50:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529166  13 57335 60611175 A  ERRORERROR'
2015-11-09 16:50:11 [DEBUG   ] Got command RPT from MCS: ref #529167
2015-11-09 16:50:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:50:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529167  77 57335 60611379 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:50:11 [DEBUG   ] Got command RPT from MCS: ref #529168
2015-11-09 16:50:11 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 16:50:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529168  12 57335 60611583 A  ERRORB‹ÌÍ'
2015-11-09 16:50:30 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 16:50:50 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 16:51:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:51:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:51:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:51:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:51:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:51:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:51:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:51:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:51:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:51:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:51:07 [DEBUG   ] Finished updating board flags in 1.598 seconds
2015-11-09 16:51:10 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 16:51:30 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 16:51:50 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 16:52:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:52:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:52:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:52:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:52:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:52:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:52:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:52:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:52:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:52:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:52:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 16:52:07 [DEBUG   ] Finished updating board flags in 1.406 seconds
2015-11-09 16:52:10 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 16:52:30 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 16:52:48 [DEBUG   ] Got command RPT from MCS: ref #529172
2015-11-09 16:52:48 [DEBUG   ] summary = ERROR
2015-11-09 16:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529172  13 57335 60768587 A  ERRORERROR'
2015-11-09 16:52:48 [DEBUG   ] Got command RPT from MCS: ref #529173
2015-11-09 16:52:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529173  77 57335 60768587 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:52:50 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 16:53:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:53:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:53:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:53:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:53:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:53:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:53:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:53:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:53:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:53:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:53:07 [DEBUG   ] Finished updating board flags in 1.393 seconds
2015-11-09 16:53:11 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 16:53:31 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-09 16:53:51 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 16:54:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999019
2015-11-09 16:54:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999019
2015-11-09 16:54:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999019
2015-11-09 16:54:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999019
2015-11-09 16:54:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999019
2015-11-09 16:54:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999019
2015-11-09 16:54:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999019
2015-11-09 16:54:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999019
2015-11-09 16:54:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999019
2015-11-09 16:54:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999019
2015-11-09 16:54:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 16:54:08 [DEBUG   ] Finished updating board flags in 2.213 seconds
2015-11-09 16:54:11 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 16:54:31 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 16:54:51 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998921
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998921
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998921
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998921
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998921
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-09 16:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998921
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998921
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998921
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998921
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998921
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-09 16:55:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-09 16:55:08 [DEBUG   ] Finished updating board flags in 2.220 seconds
2015-11-09 16:55:10 [DEBUG   ] Got command RPT from MCS: ref #529193
2015-11-09 16:55:10 [DEBUG   ] summary = ERROR
2015-11-09 16:55:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529193  13 57335 60910883 A  ERRORERROR'
2015-11-09 16:55:11 [DEBUG   ] Got command RPT from MCS: ref #529194
2015-11-09 16:55:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:55:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529194  77 57335 60911075 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:55:11 [DEBUG   ] Got command RPT from MCS: ref #529195
2015-11-09 16:55:11 [DEBUG   ] summary = ERROR
2015-11-09 16:55:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529195  13 57335 60911279 A  ERRORERROR'
2015-11-09 16:55:11 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 16:55:11 [DEBUG   ] Got command RPT from MCS: ref #529196
2015-11-09 16:55:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:55:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529196  77 57335 60911483 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:55:11 [DEBUG   ] Got command RPT from MCS: ref #529197
2015-11-09 16:55:11 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 16:55:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529197  12 57335 60911675 A  ERRORBŒÌÍ'
2015-11-09 16:55:31 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 16:55:51 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998921
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998921
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998921
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998921
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998921
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 16:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998921
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998921
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998921
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998921
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998921
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 16:56:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 16:56:08 [CRITICAL] MGT underflow detected on beam(s) 1
2015-11-09 16:56:08 [DEBUG   ] Finished updating board flags in 2.180 seconds
2015-11-09 16:56:11 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 16:56:31 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 16:56:51 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 16:57:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998921
2015-11-09 16:57:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998921
2015-11-09 16:57:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998921
2015-11-09 16:57:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998921
2015-11-09 16:57:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998921
2015-11-09 16:57:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998921
2015-11-09 16:57:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998921
2015-11-09 16:57:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998921
2015-11-09 16:57:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998921
2015-11-09 16:57:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998921
2015-11-09 16:57:08 [DEBUG   ] Finished updating board flags in 2.011 seconds
2015-11-09 16:57:11 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 16:57:32 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2015-11-09 16:57:48 [DEBUG   ] Got command RPT from MCS: ref #529206
2015-11-09 16:57:48 [DEBUG   ] summary = ERROR
2015-11-09 16:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529206  13 57335 61068591 A  ERRORERROR'
2015-11-09 16:57:48 [DEBUG   ] Got command RPT from MCS: ref #529207
2015-11-09 16:57:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 16:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529207  77 57335 61068591 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 16:57:52 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 16:58:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998921
2015-11-09 16:58:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998921
2015-11-09 16:58:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998921
2015-11-09 16:58:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998921
2015-11-09 16:58:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998921
2015-11-09 16:58:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998921
2015-11-09 16:58:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998921
2015-11-09 16:58:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998921
2015-11-09 16:58:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998921
2015-11-09 16:58:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998921
2015-11-09 16:58:08 [DEBUG   ] Finished updating board flags in 1.983 seconds
2015-11-09 16:58:12 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 16:58:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 16:58:52 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-09 16:59:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998921
2015-11-09 16:59:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998921
2015-11-09 16:59:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998921
2015-11-09 16:59:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998921
2015-11-09 16:59:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998921
2015-11-09 16:59:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 16:59:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998921
2015-11-09 16:59:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998921
2015-11-09 16:59:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998921
2015-11-09 16:59:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998921
2015-11-09 16:59:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998921
2015-11-09 16:59:08 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 16:59:08 [DEBUG   ] Finished updating board flags in 1.820 seconds
2015-11-09 16:59:12 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 16:59:32 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 16:59:52 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 17:00:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998921
2015-11-09 17:00:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998921
2015-11-09 17:00:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998921
2015-11-09 17:00:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998921
2015-11-09 17:00:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998921
2015-11-09 17:00:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998921
2015-11-09 17:00:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998921
2015-11-09 17:00:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998921
2015-11-09 17:00:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998921
2015-11-09 17:00:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998921
2015-11-09 17:00:08 [DEBUG   ] Finished updating board flags in 1.784 seconds
2015-11-09 17:00:10 [DEBUG   ] Got command RPT from MCS: ref #529226
2015-11-09 17:00:10 [DEBUG   ] summary = ERROR
2015-11-09 17:00:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529226  13 57335 61210955 A  ERRORERROR'
2015-11-09 17:00:11 [DEBUG   ] Got command RPT from MCS: ref #529227
2015-11-09 17:00:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:00:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529227  77 57335 61211159 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:00:11 [DEBUG   ] Got command RPT from MCS: ref #529228
2015-11-09 17:00:11 [DEBUG   ] summary = ERROR
2015-11-09 17:00:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529228  13 57335 61211367 A  ERRORERROR'
2015-11-09 17:00:11 [DEBUG   ] Got command RPT from MCS: ref #529229
2015-11-09 17:00:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:00:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529229  77 57335 61211559 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:00:11 [DEBUG   ] Got command RPT from MCS: ref #529230
2015-11-09 17:00:11 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 17:00:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529230  12 57335 61211765 A  ERRORB‹ÌÍ'
2015-11-09 17:00:12 [DEBUG   ] Finished updating ant stats in 0.795 seconds
2015-11-09 17:00:33 [DEBUG   ] Finished updating ant stats in 1.602 seconds
2015-11-09 17:00:52 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 17:01:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998921
2015-11-09 17:01:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998921
2015-11-09 17:01:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998921
2015-11-09 17:01:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998921
2015-11-09 17:01:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998921
2015-11-09 17:01:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998921
2015-11-09 17:01:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998921
2015-11-09 17:01:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998921
2015-11-09 17:01:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998921
2015-11-09 17:01:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998921
2015-11-09 17:01:08 [DEBUG   ] Finished updating board flags in 1.644 seconds
2015-11-09 17:01:12 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 17:01:32 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 17:01:53 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 17:02:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998921
2015-11-09 17:02:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998921
2015-11-09 17:02:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998921
2015-11-09 17:02:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998921
2015-11-09 17:02:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998921
2015-11-09 17:02:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998921
2015-11-09 17:02:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998921
2015-11-09 17:02:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998921
2015-11-09 17:02:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998921
2015-11-09 17:02:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998921
2015-11-09 17:02:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 17:02:08 [DEBUG   ] Finished updating board flags in 1.578 seconds
2015-11-09 17:02:13 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 17:02:33 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 17:02:48 [DEBUG   ] Got command RPT from MCS: ref #529234
2015-11-09 17:02:48 [DEBUG   ] summary = ERROR
2015-11-09 17:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529234  13 57335 61368587 A  ERRORERROR'
2015-11-09 17:02:48 [DEBUG   ] Got command RPT from MCS: ref #529235
2015-11-09 17:02:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529235  77 57335 61368587 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:02:53 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 17:03:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998921
2015-11-09 17:03:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998921
2015-11-09 17:03:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998921
2015-11-09 17:03:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998921
2015-11-09 17:03:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998921
2015-11-09 17:03:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 17:03:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998921
2015-11-09 17:03:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998921
2015-11-09 17:03:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998921
2015-11-09 17:03:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998921
2015-11-09 17:03:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998921
2015-11-09 17:03:08 [DEBUG   ] Finished updating board flags in 1.402 seconds
2015-11-09 17:03:13 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 17:03:33 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 17:03:53 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 17:04:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998921
2015-11-09 17:04:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998921
2015-11-09 17:04:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998921
2015-11-09 17:04:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998921
2015-11-09 17:04:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998921
2015-11-09 17:04:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998921
2015-11-09 17:04:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998921
2015-11-09 17:04:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998921
2015-11-09 17:04:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998921
2015-11-09 17:04:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998921
2015-11-09 17:04:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 17:04:08 [DEBUG   ] Finished updating board flags in 1.384 seconds
2015-11-09 17:04:13 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 17:04:33 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 17:04:53 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 17:05:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998921
2015-11-09 17:05:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998921
2015-11-09 17:05:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998921
2015-11-09 17:05:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998921
2015-11-09 17:05:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998921
2015-11-09 17:05:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998921
2015-11-09 17:05:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998921
2015-11-09 17:05:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998921
2015-11-09 17:05:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998921
2015-11-09 17:05:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998921
2015-11-09 17:05:08 [DEBUG   ] Finished updating board flags in 1.445 seconds
2015-11-09 17:05:11 [DEBUG   ] Got command RPT from MCS: ref #529252
2015-11-09 17:05:11 [DEBUG   ] summary = ERROR
2015-11-09 17:05:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529252  13 57335 61511039 A  ERRORERROR'
2015-11-09 17:05:11 [DEBUG   ] Got command RPT from MCS: ref #529253
2015-11-09 17:05:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:05:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529253  77 57335 61511243 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:05:11 [DEBUG   ] Got command RPT from MCS: ref #529254
2015-11-09 17:05:11 [DEBUG   ] summary = ERROR
2015-11-09 17:05:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529254  13 57335 61511447 A  ERRORERROR'
2015-11-09 17:05:11 [DEBUG   ] Got command RPT from MCS: ref #529255
2015-11-09 17:05:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:05:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529255  77 57335 61511639 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:05:11 [DEBUG   ] Got command RPT from MCS: ref #529256
2015-11-09 17:05:11 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 17:05:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529256  12 57335 61511843 A  ERRORBŠÌÍ'
2015-11-09 17:05:13 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 17:05:33 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 17:05:53 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 17:06:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998921
2015-11-09 17:06:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998921
2015-11-09 17:06:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998921
2015-11-09 17:06:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998921
2015-11-09 17:06:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998921
2015-11-09 17:06:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998921
2015-11-09 17:06:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998921
2015-11-09 17:06:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998921
2015-11-09 17:06:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998921
2015-11-09 17:06:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998921
2015-11-09 17:06:09 [DEBUG   ] Finished updating board flags in 2.206 seconds
2015-11-09 17:06:13 [DEBUG   ] Finished updating ant stats in 0.784 seconds
2015-11-09 17:06:33 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 17:06:54 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 17:07:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998921
2015-11-09 17:07:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998921
2015-11-09 17:07:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998921
2015-11-09 17:07:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998921
2015-11-09 17:07:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998921
2015-11-09 17:07:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998921
2015-11-09 17:07:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998921
2015-11-09 17:07:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998921
2015-11-09 17:07:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998921
2015-11-09 17:07:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998921
2015-11-09 17:07:09 [DEBUG   ] Finished updating board flags in 2.155 seconds
2015-11-09 17:07:14 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 17:07:34 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 17:07:48 [DEBUG   ] Got command RPT from MCS: ref #529265
2015-11-09 17:07:48 [DEBUG   ] summary = ERROR
2015-11-09 17:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529265  13 57335 61668587 A  ERRORERROR'
2015-11-09 17:07:48 [DEBUG   ] Got command RPT from MCS: ref #529266
2015-11-09 17:07:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529266  77 57335 61668587 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:07:54 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998823
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998823
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998823
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998823
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998823
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-09 17:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998823
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998823
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998823
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998823
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998823
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-09 17:08:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-09 17:08:09 [DEBUG   ] Finished updating board flags in 2.038 seconds
2015-11-09 17:08:14 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 17:08:34 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 17:08:54 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998823
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998823
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998823
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998823
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998823
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 17:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998823
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998823
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998823
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998823
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998823
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 17:09:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 17:09:09 [DEBUG   ] Finished updating board flags in 2.034 seconds
2015-11-09 17:09:14 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 17:09:34 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 17:09:54 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 17:10:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998823
2015-11-09 17:10:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998823
2015-11-09 17:10:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998823
2015-11-09 17:10:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998823
2015-11-09 17:10:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998823
2015-11-09 17:10:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998823
2015-11-09 17:10:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998823
2015-11-09 17:10:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998823
2015-11-09 17:10:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998823
2015-11-09 17:10:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998823
2015-11-09 17:10:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 17:10:09 [DEBUG   ] Finished updating board flags in 1.963 seconds
2015-11-09 17:10:11 [DEBUG   ] Got command RPT from MCS: ref #529285
2015-11-09 17:10:11 [DEBUG   ] summary = ERROR
2015-11-09 17:10:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529285  13 57335 61811131 A  ERRORERROR'
2015-11-09 17:10:11 [DEBUG   ] Got command RPT from MCS: ref #529286
2015-11-09 17:10:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:10:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529286  77 57335 61811335 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:10:11 [DEBUG   ] Got command RPT from MCS: ref #529287
2015-11-09 17:10:11 [DEBUG   ] summary = ERROR
2015-11-09 17:10:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529287  13 57335 61811527 A  ERRORERROR'
2015-11-09 17:10:11 [DEBUG   ] Got command RPT from MCS: ref #529288
2015-11-09 17:10:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:10:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529288  77 57335 61811731 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:10:11 [DEBUG   ] Got command RPT from MCS: ref #529289
2015-11-09 17:10:11 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 17:10:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529289  12 57335 61811935 A  ERRORBŒÌÍ'
2015-11-09 17:10:14 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 17:10:34 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 17:10:54 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 17:11:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998823
2015-11-09 17:11:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998823
2015-11-09 17:11:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998823
2015-11-09 17:11:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998823
2015-11-09 17:11:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998823
2015-11-09 17:11:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998823
2015-11-09 17:11:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998823
2015-11-09 17:11:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998823
2015-11-09 17:11:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998823
2015-11-09 17:11:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998823
2015-11-09 17:11:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 17:11:09 [DEBUG   ] Finished updating board flags in 1.789 seconds
2015-11-09 17:11:14 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 17:11:34 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 17:11:54 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 17:12:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998823
2015-11-09 17:12:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998823
2015-11-09 17:12:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998823
2015-11-09 17:12:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998823
2015-11-09 17:12:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998823
2015-11-09 17:12:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998823
2015-11-09 17:12:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998823
2015-11-09 17:12:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998823
2015-11-09 17:12:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998823
2015-11-09 17:12:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998823
2015-11-09 17:12:09 [DEBUG   ] Finished updating board flags in 1.595 seconds
2015-11-09 17:12:14 [DEBUG   ] Finished updating ant stats in 0.581 seconds
2015-11-09 17:12:35 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2015-11-09 17:12:48 [DEBUG   ] Got command RPT from MCS: ref #529293
2015-11-09 17:12:48 [DEBUG   ] summary = ERROR
2015-11-09 17:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529293  13 57335 61968587 A  ERRORERROR'
2015-11-09 17:12:48 [DEBUG   ] Got command RPT from MCS: ref #529294
2015-11-09 17:12:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529294  77 57335 61968587 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:12:55 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998725
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998725
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998725
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998725
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998725
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-09 17:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998725
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998725
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998725
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998725
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998725
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-09 17:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-09 17:13:09 [DEBUG   ] Finished updating board flags in 1.427 seconds
2015-11-09 17:13:15 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 17:13:35 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 17:13:55 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 17:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 17:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 17:14:09 [DEBUG   ] Finished updating board flags in 1.384 seconds
2015-11-09 17:14:15 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 17:14:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 17:14:55 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 17:15:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:15:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:15:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:15:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:15:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:15:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 17:15:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:15:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:15:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:15:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:15:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:15:10 [DEBUG   ] Finished updating board flags in 2.229 seconds
2015-11-09 17:15:11 [DEBUG   ] Got command RPT from MCS: ref #529314
2015-11-09 17:15:11 [DEBUG   ] summary = ERROR
2015-11-09 17:15:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529314  13 57335 62111223 A  ERRORERROR'
2015-11-09 17:15:11 [DEBUG   ] Got command RPT from MCS: ref #529315
2015-11-09 17:15:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:15:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529315  77 57335 62111427 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:15:11 [DEBUG   ] Got command RPT from MCS: ref #529316
2015-11-09 17:15:11 [DEBUG   ] summary = ERROR
2015-11-09 17:15:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529316  13 57335 62111619 A  ERRORERROR'
2015-11-09 17:15:11 [DEBUG   ] Got command RPT from MCS: ref #529317
2015-11-09 17:15:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:15:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529317  77 57335 62111823 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:15:12 [DEBUG   ] Got command RPT from MCS: ref #529318
2015-11-09 17:15:12 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 17:15:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529318  12 57335 62112027 A  ERRORB�ÌÍ'
2015-11-09 17:15:15 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 17:15:35 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 17:15:55 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 17:16:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:16:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:16:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:16:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:16:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:16:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:16:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:16:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:16:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:16:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:16:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 17:16:10 [DEBUG   ] Finished updating board flags in 2.166 seconds
2015-11-09 17:16:15 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 17:16:35 [DEBUG   ] Finished updating ant stats in 0.595 seconds
2015-11-09 17:16:56 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 17:17:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:17:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:17:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:17:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:17:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:17:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:17:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:17:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:17:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:17:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:17:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 17:17:10 [DEBUG   ] Finished updating board flags in 2.019 seconds
2015-11-09 17:17:16 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 17:17:36 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 17:17:48 [DEBUG   ] Got command RPT from MCS: ref #529327
2015-11-09 17:17:48 [DEBUG   ] summary = ERROR
2015-11-09 17:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529327  13 57335 62268595 A  ERRORERROR'
2015-11-09 17:17:48 [DEBUG   ] Got command RPT from MCS: ref #529328
2015-11-09 17:17:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529328  77 57335 62268595 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:17:56 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 17:18:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:18:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:18:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:18:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:18:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:18:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:18:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:18:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:18:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:18:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:18:10 [DEBUG   ] Finished updating board flags in 1.993 seconds
2015-11-09 17:18:16 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 17:18:36 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 17:18:56 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 17:19:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:19:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:19:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:19:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:19:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:19:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:19:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:19:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:19:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:19:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:19:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 17:19:10 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 17:19:10 [DEBUG   ] Finished updating board flags in 1.806 seconds
2015-11-09 17:19:16 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 17:19:36 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 17:19:56 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-09 17:20:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:20:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:20:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:20:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:20:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:20:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:20:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:20:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:20:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:20:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:20:10 [DEBUG   ] Finished updating board flags in 1.825 seconds
2015-11-09 17:20:11 [DEBUG   ] Got command RPT from MCS: ref #529344
2015-11-09 17:20:11 [DEBUG   ] summary = ERROR
2015-11-09 17:20:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529344  13 57335 62411315 A  ERRORERROR'
2015-11-09 17:20:11 [DEBUG   ] Got command RPT from MCS: ref #529345
2015-11-09 17:20:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:20:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529345  77 57335 62411519 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:20:11 [DEBUG   ] Got command RPT from MCS: ref #529346
2015-11-09 17:20:11 [DEBUG   ] summary = ERROR
2015-11-09 17:20:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529346  13 57335 62411723 A  ERRORERROR'
2015-11-09 17:20:11 [DEBUG   ] Got command RPT from MCS: ref #529347
2015-11-09 17:20:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:20:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529347  77 57335 62411927 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:20:12 [DEBUG   ] Got command RPT from MCS: ref #529348
2015-11-09 17:20:12 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 17:20:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529348  12 57335 62412119 A  ERRORB‹ÌÍ'
2015-11-09 17:20:16 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 17:20:36 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 17:20:56 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 17:21:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:21:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:21:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:21:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:21:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:21:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:21:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:21:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:21:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:21:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:21:10 [DEBUG   ] Finished updating board flags in 1.771 seconds
2015-11-09 17:21:16 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 17:21:36 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 17:21:57 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 17:22:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:22:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:22:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:22:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:22:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:22:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:22:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:22:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:22:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:22:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:22:10 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 17:22:10 [DEBUG   ] Finished updating board flags in 1.614 seconds
2015-11-09 17:22:17 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 17:22:37 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 17:22:48 [DEBUG   ] Got command RPT from MCS: ref #529352
2015-11-09 17:22:48 [DEBUG   ] summary = ERROR
2015-11-09 17:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529352  13 57335 62568595 A  ERRORERROR'
2015-11-09 17:22:48 [DEBUG   ] Got command RPT from MCS: ref #529353
2015-11-09 17:22:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529353  77 57335 62568595 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:22:57 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 17:23:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:23:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:23:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:23:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:23:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:23:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:23:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:23:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:23:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:23:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:23:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 17:23:10 [DEBUG   ] Finished updating board flags in 1.583 seconds
2015-11-09 17:23:17 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 17:23:37 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 17:23:57 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 17:24:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:24:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:24:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:24:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:24:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:24:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:24:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:24:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:24:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:24:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:24:10 [DEBUG   ] Finished updating board flags in 1.412 seconds
2015-11-09 17:24:17 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 17:24:37 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 17:24:57 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 17:25:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:25:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:25:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:25:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:25:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:25:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:25:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:25:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:25:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:25:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:25:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 17:25:10 [DEBUG   ] Finished updating board flags in 1.389 seconds
2015-11-09 17:25:11 [DEBUG   ] Got command RPT from MCS: ref #529373
2015-11-09 17:25:11 [DEBUG   ] summary = ERROR
2015-11-09 17:25:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529373  13 57335 62711399 A  ERRORERROR'
2015-11-09 17:25:11 [DEBUG   ] Got command RPT from MCS: ref #529374
2015-11-09 17:25:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:25:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529374  77 57335 62711603 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:25:11 [DEBUG   ] Got command RPT from MCS: ref #529375
2015-11-09 17:25:11 [DEBUG   ] summary = ERROR
2015-11-09 17:25:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529375  13 57335 62711807 A  ERRORERROR'
2015-11-09 17:25:12 [DEBUG   ] Got command RPT from MCS: ref #529376
2015-11-09 17:25:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:25:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529376  77 57335 62712011 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:25:12 [DEBUG   ] Got command RPT from MCS: ref #529377
2015-11-09 17:25:12 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 17:25:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529377  12 57335 62712203 A  ERRORB‹ÌÍ'
2015-11-09 17:25:17 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-09 17:25:37 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 17:25:57 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 17:26:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:26:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:26:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:26:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:26:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:26:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:26:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:26:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:26:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:26:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:26:11 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 17:26:11 [DEBUG   ] Finished updating board flags in 2.211 seconds
2015-11-09 17:26:17 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 17:26:37 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 17:26:57 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 17:27:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:27:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:27:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:27:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:27:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:27:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 17:27:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:27:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:27:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:27:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:27:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:27:11 [DEBUG   ] Finished updating board flags in 2.181 seconds
2015-11-09 17:27:17 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 17:27:38 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-09 17:27:48 [DEBUG   ] Got command RPT from MCS: ref #529386
2015-11-09 17:27:48 [DEBUG   ] summary = ERROR
2015-11-09 17:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529386  13 57335 62868595 A  ERRORERROR'
2015-11-09 17:27:48 [DEBUG   ] Got command RPT from MCS: ref #529387
2015-11-09 17:27:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529387  77 57335 62868595 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:27:58 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 17:28:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:28:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:28:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:28:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:28:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:28:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 17:28:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:28:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:28:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:28:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:28:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:28:11 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 17:28:11 [DEBUG   ] Finished updating board flags in 2.069 seconds
2015-11-09 17:28:18 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 17:28:38 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 17:28:58 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 17:29:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:29:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:29:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:29:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:29:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:29:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:29:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:29:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:29:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:29:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:29:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 17:29:11 [DEBUG   ] Finished updating board flags in 1.933 seconds
2015-11-09 17:29:18 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 17:29:38 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 17:29:58 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 17:30:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:30:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:30:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:30:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:30:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:30:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:30:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:30:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:30:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:30:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:30:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 17:30:11 [DEBUG   ] Got command RPT from MCS: ref #529406
2015-11-09 17:30:11 [DEBUG   ] summary = ERROR
2015-11-09 17:30:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529406  13 57335 63011751 A  ERRORERROR'
2015-11-09 17:30:11 [DEBUG   ] Got command RPT from MCS: ref #529407
2015-11-09 17:30:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:30:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529407  77 57335 63011908 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:30:11 [DEBUG   ] Finished updating board flags in 2.109 seconds
2015-11-09 17:30:12 [DEBUG   ] Got command RPT from MCS: ref #529408
2015-11-09 17:30:12 [DEBUG   ] summary = ERROR
2015-11-09 17:30:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529408  13 57335 63012099 A  ERRORERROR'
2015-11-09 17:30:12 [DEBUG   ] Got command RPT from MCS: ref #529409
2015-11-09 17:30:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:30:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529409  77 57335 63012303 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:30:12 [DEBUG   ] Got command RPT from MCS: ref #529410
2015-11-09 17:30:12 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 17:30:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529410  12 57335 63012507 A  ERRORBŠÌÍ'
2015-11-09 17:30:18 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 17:30:38 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 17:30:58 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 17:31:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:31:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:31:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:31:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:31:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:31:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:31:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:31:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:31:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:31:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:31:11 [DEBUG   ] Finished updating board flags in 1.695 seconds
2015-11-09 17:31:18 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 17:31:38 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-09 17:31:58 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 17:32:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:32:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:32:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:32:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:32:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:32:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:32:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:32:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:32:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:32:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:32:11 [DEBUG   ] Finished updating board flags in 1.623 seconds
2015-11-09 17:32:18 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 17:32:38 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 17:32:48 [DEBUG   ] Got command RPT from MCS: ref #529414
2015-11-09 17:32:48 [DEBUG   ] summary = ERROR
2015-11-09 17:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529414  13 57335 63168595 A  ERRORERROR'
2015-11-09 17:32:48 [DEBUG   ] Got command RPT from MCS: ref #529415
2015-11-09 17:32:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529415  77 57335 63168595 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:32:59 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 17:33:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:33:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:33:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:33:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:33:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:33:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:33:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:33:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:33:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:33:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:33:11 [DEBUG   ] Finished updating board flags in 1.586 seconds
2015-11-09 17:33:19 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 17:33:39 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 17:33:59 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 17:34:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:34:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:34:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:34:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:34:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:34:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:34:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:34:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:34:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:34:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:34:11 [DEBUG   ] Finished updating board flags in 1.411 seconds
2015-11-09 17:34:19 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 17:34:39 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 17:34:59 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 17:35:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:35:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:35:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:35:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:35:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:35:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:35:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:35:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:35:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:35:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:35:11 [DEBUG   ] Got command RPT from MCS: ref #529432
2015-11-09 17:35:11 [DEBUG   ] summary = ERROR
2015-11-09 17:35:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529432  13 57335 63311583 A  ERRORERROR'
2015-11-09 17:35:11 [DEBUG   ] Finished updating board flags in 1.461 seconds
2015-11-09 17:35:11 [DEBUG   ] Got command RPT from MCS: ref #529433
2015-11-09 17:35:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:35:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529433  77 57335 63311791 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:35:11 [DEBUG   ] Got command RPT from MCS: ref #529434
2015-11-09 17:35:11 [DEBUG   ] summary = ERROR
2015-11-09 17:35:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529434  13 57335 63311983 A  ERRORERROR'
2015-11-09 17:35:12 [DEBUG   ] Got command RPT from MCS: ref #529435
2015-11-09 17:35:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:35:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529435  77 57335 63312187 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:35:12 [DEBUG   ] Got command RPT from MCS: ref #529436
2015-11-09 17:35:12 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 17:35:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529436  12 57335 63312391 A  ERRORB‹ÌÍ'
2015-11-09 17:35:19 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 17:35:39 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 17:35:59 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 17:36:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:36:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:36:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:36:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:36:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:36:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:36:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:36:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:36:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:36:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:36:11 [DEBUG   ] Finished updating board flags in 1.459 seconds
2015-11-09 17:36:19 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 17:36:39 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 17:36:59 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-09 17:37:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:37:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:37:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:37:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:37:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:37:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 17:37:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:37:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:37:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:37:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:37:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:37:11 [DEBUG   ] Finished updating board flags in 1.460 seconds
2015-11-09 17:37:19 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 17:37:39 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 17:37:48 [DEBUG   ] Got command RPT from MCS: ref #529445
2015-11-09 17:37:48 [DEBUG   ] summary = ERROR
2015-11-09 17:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529445  13 57335 63468599 A  ERRORERROR'
2015-11-09 17:37:48 [DEBUG   ] Got command RPT from MCS: ref #529446
2015-11-09 17:37:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529446  77 57335 63468599 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:38:00 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 17:38:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:38:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:38:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:38:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:38:10 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:38:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:38:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:38:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:38:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:38:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:38:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa34, chip 1
2015-11-09 17:38:11 [DEBUG   ] Finished updating board flags in 1.460 seconds
2015-11-09 17:38:20 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 17:38:40 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 17:39:00 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 17:39:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:39:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:39:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:39:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:39:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:39:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:39:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:39:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:39:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:39:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:39:12 [DEBUG   ] Finished updating board flags in 2.210 seconds
2015-11-09 17:39:20 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 17:39:40 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 17:40:00 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 17:40:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:40:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:40:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:40:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:40:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:40:11 [DEBUG   ] Got command RPT from MCS: ref #529465
2015-11-09 17:40:11 [DEBUG   ] summary = ERROR
2015-11-09 17:40:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529465  13 57335 63611679 A  ERRORERROR'
2015-11-09 17:40:11 [DEBUG   ] Got command RPT from MCS: ref #529466
2015-11-09 17:40:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:40:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529466  77 57335 63611871 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:40:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:40:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:40:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:40:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:40:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:40:12 [DEBUG   ] Got command RPT from MCS: ref #529467
2015-11-09 17:40:12 [DEBUG   ] summary = ERROR
2015-11-09 17:40:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529467  13 57335 63612071 A  ERRORERROR'
2015-11-09 17:40:12 [DEBUG   ] Got command RPT from MCS: ref #529468
2015-11-09 17:40:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:40:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529468  77 57335 63612279 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:40:12 [DEBUG   ] Got command RPT from MCS: ref #529469
2015-11-09 17:40:12 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 17:40:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529469  12 57335 63612471 A  ERRORBŒÌÍ'
2015-11-09 17:40:12 [DEBUG   ] Finished updating board flags in 2.142 seconds
2015-11-09 17:40:20 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 17:40:40 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 17:41:00 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 17:41:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:41:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:41:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:41:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:41:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:41:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:41:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:41:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:41:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:41:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:41:12 [DEBUG   ] Finished updating board flags in 2.040 seconds
2015-11-09 17:41:20 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 17:41:40 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 17:42:00 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 17:42:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:42:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:42:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:42:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:42:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:42:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:42:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:42:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:42:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:42:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:42:12 [DEBUG   ] Finished updating board flags in 2.008 seconds
2015-11-09 17:42:20 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 17:42:40 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 17:42:48 [DEBUG   ] Got command RPT from MCS: ref #529473
2015-11-09 17:42:48 [DEBUG   ] summary = ERROR
2015-11-09 17:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529473  13 57335 63768591 A  ERRORERROR'
2015-11-09 17:42:48 [DEBUG   ] Got command RPT from MCS: ref #529474
2015-11-09 17:42:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529474  77 57335 63768591 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:43:00 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2015-11-09 17:43:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:43:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:43:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:43:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:43:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:43:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:43:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:43:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:43:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:43:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:43:12 [DEBUG   ] Finished updating board flags in 1.960 seconds
2015-11-09 17:43:20 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 17:43:40 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 17:44:01 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 17:44:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:44:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:44:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:44:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:44:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:44:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:44:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:44:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:44:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:44:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:44:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 17:44:12 [DEBUG   ] Finished updating board flags in 1.816 seconds
2015-11-09 17:44:21 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 17:44:41 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 17:45:01 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 17:45:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:45:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:45:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:45:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:45:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:45:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 17:45:11 [DEBUG   ] Got command RPT from MCS: ref #529494
2015-11-09 17:45:11 [DEBUG   ] summary = ERROR
2015-11-09 17:45:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529494  13 57335 63911751 A  ERRORERROR'
2015-11-09 17:45:11 [DEBUG   ] Got command RPT from MCS: ref #529495
2015-11-09 17:45:11 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:45:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529495  77 57335 63911955 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:45:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:45:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:45:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:45:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:45:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:45:12 [DEBUG   ] Got command RPT from MCS: ref #529496
2015-11-09 17:45:12 [DEBUG   ] summary = ERROR
2015-11-09 17:45:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529496  13 57335 63912159 A  ERRORERROR'
2015-11-09 17:45:12 [DEBUG   ] Got command RPT from MCS: ref #529497
2015-11-09 17:45:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:45:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529497  77 57335 63912359 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:45:12 [DEBUG   ] Got command RPT from MCS: ref #529498
2015-11-09 17:45:12 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 17:45:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529498  12 57335 63912555 A  ERRORB‹ÌÍ'
2015-11-09 17:45:12 [DEBUG   ] Finished updating board flags in 1.781 seconds
2015-11-09 17:45:21 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 17:45:41 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 17:46:01 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 17:46:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:46:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:46:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:46:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:46:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:46:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:46:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:46:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:46:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:46:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:46:12 [DEBUG   ] Finished updating board flags in 1.623 seconds
2015-11-09 17:46:21 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 17:46:41 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 17:47:01 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 17:47:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:47:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:47:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:47:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:47:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:47:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:47:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:47:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:47:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:47:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:47:12 [DEBUG   ] Finished updating board flags in 1.592 seconds
2015-11-09 17:47:21 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2015-11-09 17:47:41 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 17:47:48 [DEBUG   ] Got command RPT from MCS: ref #529507
2015-11-09 17:47:48 [DEBUG   ] summary = ERROR
2015-11-09 17:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529507  13 57335 64068599 A  ERRORERROR'
2015-11-09 17:47:48 [DEBUG   ] Got command RPT from MCS: ref #529508
2015-11-09 17:47:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529508  77 57335 64068599 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:48:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 17:48:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:48:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:48:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:48:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:48:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:48:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:48:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:48:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:48:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:48:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:48:12 [DEBUG   ] Finished updating board flags in 1.412 seconds
2015-11-09 17:48:21 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 17:48:41 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 17:49:01 [DEBUG   ] Finished updating ant stats in 0.607 seconds
2015-11-09 17:49:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:49:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:49:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:49:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:49:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:49:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:49:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:49:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:49:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:49:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:49:12 [DEBUG   ] Finished updating board flags in 1.411 seconds
2015-11-09 17:49:21 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 17:49:41 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 17:50:02 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 17:50:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:50:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:50:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:50:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:50:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:50:11 [DEBUG   ] Got command RPT from MCS: ref #529524
2015-11-09 17:50:11 [DEBUG   ] summary = ERROR
2015-11-09 17:50:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529524  13 57335 64211847 A  ERRORERROR'
2015-11-09 17:50:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:50:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:50:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:50:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:50:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:50:12 [DEBUG   ] Got command RPT from MCS: ref #529525
2015-11-09 17:50:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:50:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529525  77 57335 64212051 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:50:12 [DEBUG   ] Got command RPT from MCS: ref #529526
2015-11-09 17:50:12 [DEBUG   ] summary = ERROR
2015-11-09 17:50:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529526  13 57335 64212255 A  ERRORERROR'
2015-11-09 17:50:12 [DEBUG   ] Got command RPT from MCS: ref #529527
2015-11-09 17:50:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:50:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529527  77 57335 64212447 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:50:12 [DEBUG   ] Finished updating board flags in 1.380 seconds
2015-11-09 17:50:12 [DEBUG   ] Got command RPT from MCS: ref #529528
2015-11-09 17:50:12 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 17:50:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529528  12 57335 64212651 A  ERRORBŒÌÍ'
2015-11-09 17:50:22 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 17:50:42 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 17:51:02 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 17:51:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:51:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:51:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:51:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:51:11 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:51:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:51:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:51:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:51:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:51:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:51:12 [DEBUG   ] Finished updating board flags in 1.388 seconds
2015-11-09 17:51:22 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 17:51:42 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 17:52:02 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 17:52:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:52:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:52:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:52:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:52:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:52:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:52:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:52:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:52:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:52:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:52:13 [DEBUG   ] Finished updating board flags in 2.037 seconds
2015-11-09 17:52:22 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 17:52:42 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 17:52:48 [DEBUG   ] Got command RPT from MCS: ref #529532
2015-11-09 17:52:48 [DEBUG   ] summary = ERROR
2015-11-09 17:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529532  13 57335 64368595 A  ERRORERROR'
2015-11-09 17:52:48 [DEBUG   ] Got command RPT from MCS: ref #529533
2015-11-09 17:52:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529533  77 57335 64368595 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:53:02 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 17:53:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:53:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:53:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:53:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:53:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:53:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:53:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:53:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:53:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:53:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:53:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 17:53:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2015-11-09 17:53:13 [CRITICAL] MGT underflow detected on beam(s) 1
2015-11-09 17:53:13 [DEBUG   ] Finished updating board flags in 1.986 seconds
2015-11-09 17:53:22 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 17:53:42 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 17:54:02 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 17:54:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:54:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:54:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:54:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:54:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:54:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:54:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:54:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:54:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:54:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:54:13 [DEBUG   ] Finished updating board flags in 1.814 seconds
2015-11-09 17:54:22 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 17:54:42 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 17:55:03 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 17:55:11 [DEBUG   ] Got command RPT from MCS: ref #529553
2015-11-09 17:55:11 [DEBUG   ] summary = ERROR
2015-11-09 17:55:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529553  13 57335 64511947 A  ERRORERROR'
2015-11-09 17:55:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:55:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:55:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:55:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:55:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:55:12 [DEBUG   ] Got command RPT from MCS: ref #529554
2015-11-09 17:55:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:55:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529554  77 57335 64512151 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:55:12 [DEBUG   ] Got command RPT from MCS: ref #529555
2015-11-09 17:55:12 [DEBUG   ] summary = ERROR
2015-11-09 17:55:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529555  13 57335 64512355 A  ERRORERROR'
2015-11-09 17:55:12 [DEBUG   ] Got command RPT from MCS: ref #529556
2015-11-09 17:55:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:55:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529556  77 57335 64512551 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:55:12 [DEBUG   ] Got command RPT from MCS: ref #529557
2015-11-09 17:55:12 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 17:55:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529557  12 57335 64512755 A  ERRORB�'
2015-11-09 17:55:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:55:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:55:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:55:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:55:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:55:13 [DEBUG   ] Finished updating board flags in 1.784 seconds
2015-11-09 17:55:23 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-09 17:55:43 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 17:56:03 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 17:56:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:56:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:56:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:56:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:56:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:56:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:56:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:56:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:56:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:56:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:56:13 [DEBUG   ] Finished updating board flags in 1.621 seconds
2015-11-09 17:56:23 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 17:56:43 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 17:57:03 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2015-11-09 17:57:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:57:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:57:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:57:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:57:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:57:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:57:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:57:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:57:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:57:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:57:13 [DEBUG   ] Finished updating board flags in 1.586 seconds
2015-11-09 17:57:23 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 17:57:43 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 17:57:48 [DEBUG   ] Got command RPT from MCS: ref #529566
2015-11-09 17:57:48 [DEBUG   ] summary = ERROR
2015-11-09 17:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529566  13 57335 64668595 A  ERRORERROR'
2015-11-09 17:57:48 [DEBUG   ] Got command RPT from MCS: ref #529567
2015-11-09 17:57:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 17:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529567  77 57335 64668595 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 17:58:03 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 17:58:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:58:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:58:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:58:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:58:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:58:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:58:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:58:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:58:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:58:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:58:13 [DEBUG   ] Finished updating board flags in 1.413 seconds
2015-11-09 17:58:23 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 17:58:44 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 17:59:04 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-09 17:59:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 17:59:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 17:59:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 17:59:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 17:59:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 17:59:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 17:59:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 17:59:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 17:59:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 17:59:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 17:59:13 [DEBUG   ] Finished updating board flags in 1.385 seconds
2015-11-09 17:59:24 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 17:59:44 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 18:00:04 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 18:00:12 [DEBUG   ] Got command RPT from MCS: ref #529586
2015-11-09 18:00:12 [DEBUG   ] summary = ERROR
2015-11-09 18:00:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529586  13 57335 64812059 A  ERRORERROR'
2015-11-09 18:00:12 [DEBUG   ] Got command RPT from MCS: ref #529587
2015-11-09 18:00:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:00:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529587  77 57335 64812251 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:00:12 [DEBUG   ] Got command RPT from MCS: ref #529588
2015-11-09 18:00:12 [DEBUG   ] summary = ERROR
2015-11-09 18:00:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529588  13 57335 64812443 A  ERRORERROR'
2015-11-09 18:00:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:00:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:00:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:00:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:00:12 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:00:12 [DEBUG   ] Got command RPT from MCS: ref #529589
2015-11-09 18:00:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:00:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529589  77 57335 64812651 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:00:12 [DEBUG   ] Got command RPT from MCS: ref #529590
2015-11-09 18:00:12 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 18:00:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529590  12 57335 64812843 A  ERRORBr'
2015-11-09 18:00:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:00:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:00:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:00:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:00:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:00:13 [DEBUG   ] Finished updating board flags in 1.377 seconds
2015-11-09 18:00:24 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-09 18:00:44 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 18:01:04 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 18:01:11 [DEBUG   ] Got command TBW from MCS: ref #529592
2015-11-09 18:01:11 [DEBUG   ] tbw = exited with status True, code 0x00
2015-11-09 18:01:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBW   529592   8 57335 64871247 A  ERROR'
2015-11-09 18:01:12 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 pkill tbn
2015-11-09 18:01:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:01:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:01:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:01:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:01:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 pkill tbn
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 pkill tbn
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 pkill tbn
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 pkill tbn
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 pkill tbn
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 pkill tbn
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 pkill tbn
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 pkill tbn
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 pkill tbn
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 pkill tbn
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 pkill tbn
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 pkill tbn
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 pkill tbn
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 pkill tbn
2015-11-09 18:01:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 pkill tbn
2015-11-09 18:01:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 pkill tbn
2015-11-09 18:01:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:01:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:01:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:01:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:01:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:01:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 pkill tbn
2015-11-09 18:01:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 pkill tbn
2015-11-09 18:01:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 pkill tbn
2015-11-09 18:01:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 pkill tbn
2015-11-09 18:01:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 pkill tbn
2015-11-09 18:01:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 pkill tbn
2015-11-09 18:01:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 pkill tbn
2015-11-09 18:01:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 pkill tbn
2015-11-09 18:01:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 pkill tbn
2015-11-09 18:01:14 [DEBUG   ] Finished updating board flags in 2.070 seconds
2015-11-09 18:01:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 pkill tbn
2015-11-09 18:01:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 pkill tbn
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa27', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '1', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa24', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '2', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa23', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '3', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa18', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '4', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa10', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '5', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa32', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '6', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa11', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '7', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa30', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '8', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa12', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '9', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa8', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '10', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa13', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '11', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa26', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '12', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa14', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '13', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa22', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '14', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa20', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '15', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa1', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '16', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa5', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '17', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa25', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '18', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa35', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '19', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa19', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '20', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa21', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '21', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa6', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '22', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa7', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '23', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa33', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '24', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa15', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '25', '12000000']
2015-11-09 18:01:14 [DEBUG   ] spawn_process: at 2015-11-09 18:01:14 running ['/usr/local/bin/lwa28', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '26', '12000000']
2015-11-09 18:01:14 [INFO    ] Finished spawning TBW
2015-11-09 18:02:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:02:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:02:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:02:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:02:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:02:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:02:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:02:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:02:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:02:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:02:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 18:02:14 [DEBUG   ] Finished updating board flags in 1.887 seconds
2015-11-09 18:02:48 [DEBUG   ] Got command RPT from MCS: ref #529600
2015-11-09 18:02:48 [DEBUG   ] summary = ERROR
2015-11-09 18:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529600  13 57335 64968603 A  ERRORERROR'
2015-11-09 18:02:48 [DEBUG   ] Got command RPT from MCS: ref #529601
2015-11-09 18:02:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529601  77 57335 64968603 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:03:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:03:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:03:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:03:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:03:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:03:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 18:03:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:03:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:03:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:03:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:03:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:03:14 [DEBUG   ] Finished updating board flags in 1.706 seconds
2015-11-09 18:04:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:04:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:04:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:04:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:04:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:04:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:04:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:04:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:04:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:04:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:04:14 [DEBUG   ] Finished updating board flags in 1.709 seconds
2015-11-09 18:04:53 [INFO    ] __waitForTBW - TBW finished in 218.287 s
2015-11-09 18:04:54 [INFO    ] Started antennas statistics background thread
2015-11-09 18:04:54 [INFO    ] Resuming TBN at 38.100 MHz after TBW acquisition
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '1', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '2', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '3', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '4', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '5', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '6', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '7', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '8', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '9', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '10', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '11', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '12', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '13', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '14', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '15', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '16', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '17', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '18', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '19', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '20', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '21', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '22', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '23', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '24', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '25', '1447092295']
2015-11-09 18:04:54 [DEBUG   ] spawn_process: at 2015-11-09 18:04:54 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '26', '1447092295']
2015-11-09 18:04:54 [INFO    ] Finished spawning TBN in 0.057 s
2015-11-09 18:04:54 [DEBUG   ] Finished updating ant stats in 1.174 seconds
2015-11-09 18:05:12 [DEBUG   ] Got command RPT from MCS: ref #529623
2015-11-09 18:05:12 [DEBUG   ] summary = ERROR
2015-11-09 18:05:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529623  13 57335 65112143 A  ERRORERROR'
2015-11-09 18:05:12 [DEBUG   ] Got command RPT from MCS: ref #529624
2015-11-09 18:05:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:05:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529624  77 57335 65112347 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:05:12 [DEBUG   ] Got command RPT from MCS: ref #529625
2015-11-09 18:05:12 [DEBUG   ] summary = ERROR
2015-11-09 18:05:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529625  13 57335 65112551 A  ERRORERROR'
2015-11-09 18:05:12 [DEBUG   ] Got command RPT from MCS: ref #529626
2015-11-09 18:05:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:05:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529626  77 57335 65112743 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:05:12 [DEBUG   ] Got command RPT from MCS: ref #529627
2015-11-09 18:05:12 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 18:05:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529627  12 57335 65112947 A  ERRORBŠÌÍ'
2015-11-09 18:05:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:05:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:05:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:05:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:05:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:05:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:05:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:05:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:05:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:05:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:05:14 [DEBUG   ] Finished updating ant stats in 1.039 seconds
2015-11-09 18:05:14 [DEBUG   ] Finished updating board flags in 1.858 seconds
2015-11-09 18:05:34 [DEBUG   ] Finished updating ant stats in 0.949 seconds
2015-11-09 18:05:54 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 18:06:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:06:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:06:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:06:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:06:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:06:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:06:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:06:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:06:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:06:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:06:14 [DEBUG   ] Finished updating ant stats in 0.852 seconds
2015-11-09 18:06:14 [DEBUG   ] Finished updating board flags in 1.810 seconds
2015-11-09 18:06:34 [DEBUG   ] Finished updating ant stats in 0.749 seconds
2015-11-09 18:06:54 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 18:07:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:07:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:07:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:07:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:07:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:07:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:07:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:07:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:07:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:07:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:07:14 [DEBUG   ] Finished updating ant stats in 0.644 seconds
2015-11-09 18:07:14 [DEBUG   ] Finished updating board flags in 1.670 seconds
2015-11-09 18:07:34 [DEBUG   ] Finished updating ant stats in 0.559 seconds
2015-11-09 18:07:48 [DEBUG   ] Got command RPT from MCS: ref #529639
2015-11-09 18:07:48 [DEBUG   ] summary = ERROR
2015-11-09 18:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529639  13 57335 65268595 A  ERRORERROR'
2015-11-09 18:07:48 [DEBUG   ] Got command RPT from MCS: ref #529640
2015-11-09 18:07:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529640  77 57335 65268595 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:07:55 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 18:08:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:08:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:08:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:08:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:08:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:08:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:08:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:08:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:08:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:08:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:08:14 [DEBUG   ] Finished updating board flags in 1.627 seconds
2015-11-09 18:08:15 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-09 18:08:35 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 18:08:55 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 18:09:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:09:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:09:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:09:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:09:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:09:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:09:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:09:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:09:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:09:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:09:14 [DEBUG   ] Finished updating board flags in 1.539 seconds
2015-11-09 18:09:15 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 18:09:35 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 18:09:55 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 18:10:12 [DEBUG   ] Got command RPT from MCS: ref #529659
2015-11-09 18:10:12 [DEBUG   ] summary = ERROR
2015-11-09 18:10:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529659  13 57335 65412231 A  ERRORERROR'
2015-11-09 18:10:12 [DEBUG   ] Got command RPT from MCS: ref #529660
2015-11-09 18:10:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:10:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529660  77 57335 65412435 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:10:12 [DEBUG   ] Got command RPT from MCS: ref #529661
2015-11-09 18:10:12 [DEBUG   ] summary = ERROR
2015-11-09 18:10:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529661  13 57335 65412639 A  ERRORERROR'
2015-11-09 18:10:12 [DEBUG   ] Got command RPT from MCS: ref #529662
2015-11-09 18:10:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:10:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529662  77 57335 65412843 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:10:13 [DEBUG   ] Got command RPT from MCS: ref #529663
2015-11-09 18:10:13 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 18:10:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529663  12 57335 65413035 A  ERRORB�ÌÍ'
2015-11-09 18:10:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:10:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:10:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:10:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:10:13 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:10:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:10:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:10:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:10:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:10:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:10:14 [DEBUG   ] Finished updating board flags in 1.520 seconds
2015-11-09 18:10:15 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 18:10:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 18:10:55 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 18:11:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:11:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:11:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:11:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:11:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:11:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:11:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:11:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:11:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:11:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:11:15 [DEBUG   ] Finished updating ant stats in 0.847 seconds
2015-11-09 18:11:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 18:11:15 [DEBUG   ] Finished updating board flags in 2.215 seconds
2015-11-09 18:11:35 [DEBUG   ] Finished updating ant stats in 0.760 seconds
2015-11-09 18:11:55 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 18:12:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:12:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:12:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:12:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:12:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:12:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:12:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:12:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:12:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:12:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:12:15 [DEBUG   ] Finished updating ant stats in 0.643 seconds
2015-11-09 18:12:15 [DEBUG   ] Finished updating board flags in 2.211 seconds
2015-11-09 18:12:35 [DEBUG   ] Finished updating ant stats in 0.559 seconds
2015-11-09 18:12:48 [DEBUG   ] Got command RPT from MCS: ref #529667
2015-11-09 18:12:48 [DEBUG   ] summary = ERROR
2015-11-09 18:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529667  13 57335 65568607 A  ERRORERROR'
2015-11-09 18:12:48 [DEBUG   ] Got command RPT from MCS: ref #529668
2015-11-09 18:12:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529668  77 57335 65568607 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:12:56 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 18:13:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:13:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:13:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:13:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:13:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:13:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:13:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:13:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:13:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:13:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:13:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 18:13:15 [DEBUG   ] Finished updating board flags in 2.164 seconds
2015-11-09 18:13:16 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 18:13:36 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 18:13:56 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 18:14:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:14:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:14:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:14:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:14:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:14:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:14:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:14:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:14:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:14:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:14:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2015-11-09 18:14:15 [CRITICAL] MGT underflow detected on beam(s) 1
2015-11-09 18:14:15 [DEBUG   ] Finished updating board flags in 2.010 seconds
2015-11-09 18:14:16 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 18:14:36 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 18:14:56 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 18:15:12 [DEBUG   ] Got command RPT from MCS: ref #529688
2015-11-09 18:15:12 [DEBUG   ] summary = ERROR
2015-11-09 18:15:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529688  13 57335 65712323 A  ERRORERROR'
2015-11-09 18:15:12 [DEBUG   ] Got command RPT from MCS: ref #529689
2015-11-09 18:15:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:15:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529689  77 57335 65712515 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:15:12 [DEBUG   ] Got command RPT from MCS: ref #529690
2015-11-09 18:15:12 [DEBUG   ] summary = ERROR
2015-11-09 18:15:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529690  13 57335 65712719 A  ERRORERROR'
2015-11-09 18:15:12 [DEBUG   ] Got command RPT from MCS: ref #529691
2015-11-09 18:15:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:15:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529691  77 57335 65712923 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:15:13 [DEBUG   ] Got command RPT from MCS: ref #529692
2015-11-09 18:15:13 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 18:15:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529692  12 57335 65713127 A  ERRORBŒÌÍ'
2015-11-09 18:15:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:15:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:15:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:15:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:15:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:15:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:15:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:15:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:15:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:15:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:15:15 [DEBUG   ] Finished updating board flags in 2.008 seconds
2015-11-09 18:15:16 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 18:15:36 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 18:15:56 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 18:16:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:16:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:16:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:16:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:16:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:16:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 18:16:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:16:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:16:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:16:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:16:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:16:15 [DEBUG   ] Finished updating board flags in 1.957 seconds
2015-11-09 18:16:16 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 18:16:36 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 18:16:56 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 18:17:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:17:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:17:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:17:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:17:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:17:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:17:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:17:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:17:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:17:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:17:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 18:17:15 [DEBUG   ] Finished updating board flags in 1.796 seconds
2015-11-09 18:17:17 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-09 18:17:37 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 18:17:48 [DEBUG   ] Got command RPT from MCS: ref #529701
2015-11-09 18:17:48 [DEBUG   ] summary = ERROR
2015-11-09 18:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529701  13 57335 65868603 A  ERRORERROR'
2015-11-09 18:17:48 [DEBUG   ] Got command RPT from MCS: ref #529702
2015-11-09 18:17:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529702  77 57335 65868603 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:17:57 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 18:18:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:18:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:18:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:18:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:18:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:18:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:18:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:18:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:18:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:18:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:18:15 [DEBUG   ] Finished updating board flags in 1.611 seconds
2015-11-09 18:18:17 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 18:18:37 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 18:18:57 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 18:19:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:19:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:19:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:19:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:19:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:19:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:19:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:19:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:19:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:19:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:19:15 [DEBUG   ] Finished updating board flags in 1.589 seconds
2015-11-09 18:19:17 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2015-11-09 18:19:37 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 18:19:57 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 18:20:12 [DEBUG   ] Got command RPT from MCS: ref #529718
2015-11-09 18:20:12 [DEBUG   ] summary = ERROR
2015-11-09 18:20:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529718  13 57335 66012403 A  ERRORERROR'
2015-11-09 18:20:12 [DEBUG   ] Got command RPT from MCS: ref #529719
2015-11-09 18:20:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:20:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529719  77 57335 66012595 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:20:12 [DEBUG   ] Got command RPT from MCS: ref #529720
2015-11-09 18:20:12 [DEBUG   ] summary = ERROR
2015-11-09 18:20:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529720  13 57335 66012799 A  ERRORERROR'
2015-11-09 18:20:13 [DEBUG   ] Got command RPT from MCS: ref #529721
2015-11-09 18:20:13 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:20:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529721  77 57335 66013003 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:20:13 [DEBUG   ] Got command RPT from MCS: ref #529722
2015-11-09 18:20:13 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 18:20:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529722  12 57335 66013207 A  ERRORB‹ÌÍ'
2015-11-09 18:20:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:20:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:20:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:20:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:20:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:20:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:20:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:20:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:20:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:20:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:20:15 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 18:20:15 [DEBUG   ] Finished updating board flags in 1.408 seconds
2015-11-09 18:20:17 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 18:20:37 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 18:20:57 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 18:21:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:21:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:21:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:21:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:21:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:21:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:21:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:21:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:21:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:21:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:21:15 [DEBUG   ] Finished updating board flags in 1.412 seconds
2015-11-09 18:21:18 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 18:21:38 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 18:21:58 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 18:22:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:22:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:22:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:22:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:22:14 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:22:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:22:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:22:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:22:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:22:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:22:15 [DEBUG   ] Finished updating board flags in 1.387 seconds
2015-11-09 18:22:18 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 18:22:38 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 18:22:48 [DEBUG   ] Got command RPT from MCS: ref #529726
2015-11-09 18:22:48 [DEBUG   ] summary = ERROR
2015-11-09 18:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529726  13 57335 66168599 A  ERRORERROR'
2015-11-09 18:22:48 [DEBUG   ] Got command RPT from MCS: ref #529727
2015-11-09 18:22:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529727  77 57335 66168599 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:22:58 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 18:23:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:23:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:23:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:23:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:23:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:23:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:23:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:23:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:23:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:23:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:23:16 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 18:23:16 [DEBUG   ] Finished updating board flags in 2.218 seconds
2015-11-09 18:23:18 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 18:23:38 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 18:23:58 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 18:24:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:24:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:24:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:24:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:24:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:24:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:24:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:24:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:24:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:24:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:24:16 [DEBUG   ] Finished updating board flags in 2.184 seconds
2015-11-09 18:24:18 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 18:24:38 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 18:24:58 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 18:25:12 [DEBUG   ] Got command RPT from MCS: ref #529747
2015-11-09 18:25:12 [DEBUG   ] summary = ERROR
2015-11-09 18:25:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529747  13 57335 66312475 A  ERRORERROR'
2015-11-09 18:25:12 [DEBUG   ] Got command RPT from MCS: ref #529748
2015-11-09 18:25:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:25:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529748  77 57335 66312679 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:25:12 [DEBUG   ] Got command RPT from MCS: ref #529749
2015-11-09 18:25:12 [DEBUG   ] summary = ERROR
2015-11-09 18:25:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529749  13 57335 66312883 A  ERRORERROR'
2015-11-09 18:25:13 [DEBUG   ] Got command RPT from MCS: ref #529750
2015-11-09 18:25:13 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:25:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529750  77 57335 66313087 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:25:13 [DEBUG   ] Got command RPT from MCS: ref #529751
2015-11-09 18:25:13 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 18:25:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529751  12 57335 66313279 A  ERRORBŒÌÍ'
2015-11-09 18:25:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:25:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:25:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:25:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:25:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:25:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:25:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:25:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:25:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:25:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:25:16 [DEBUG   ] Finished updating board flags in 2.010 seconds
2015-11-09 18:25:18 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 18:25:38 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 18:25:58 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 18:26:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:26:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:26:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:26:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:26:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:26:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 18:26:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:26:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:26:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:26:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:26:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:26:16 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 18:26:16 [DEBUG   ] Finished updating board flags in 1.986 seconds
2015-11-09 18:26:19 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 18:26:39 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 18:26:59 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 18:27:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:27:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:27:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:27:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:27:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:27:15 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 18:27:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:27:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:27:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:27:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:27:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:27:16 [DEBUG   ] Finished updating board flags in 1.817 seconds
2015-11-09 18:27:19 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 18:27:39 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 18:27:48 [DEBUG   ] Got command RPT from MCS: ref #529760
2015-11-09 18:27:48 [DEBUG   ] summary = ERROR
2015-11-09 18:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529760  13 57335 66468603 A  ERRORERROR'
2015-11-09 18:27:48 [DEBUG   ] Got command RPT from MCS: ref #529761
2015-11-09 18:27:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529761  77 57335 66468603 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:27:59 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 18:28:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:28:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:28:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:28:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:28:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:28:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:28:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:28:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:28:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:28:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:28:16 [DEBUG   ] Finished updating board flags in 1.788 seconds
2015-11-09 18:28:19 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 18:28:39 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2015-11-09 18:28:59 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 18:29:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:29:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:29:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:29:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:29:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:29:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:29:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:29:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:29:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:29:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:29:16 [DEBUG   ] Finished updating board flags in 1.607 seconds
2015-11-09 18:29:19 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 18:29:39 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 18:29:59 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 18:30:12 [DEBUG   ] Got command RPT from MCS: ref #529780
2015-11-09 18:30:12 [DEBUG   ] summary = ERROR
2015-11-09 18:30:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529780  13 57335 66612567 A  ERRORERROR'
2015-11-09 18:30:12 [DEBUG   ] Got command RPT from MCS: ref #529781
2015-11-09 18:30:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:30:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529781  77 57335 66612771 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:30:12 [DEBUG   ] Got command RPT from MCS: ref #529782
2015-11-09 18:30:12 [DEBUG   ] summary = ERROR
2015-11-09 18:30:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529782  13 57335 66612963 A  ERRORERROR'
2015-11-09 18:30:13 [DEBUG   ] Got command RPT from MCS: ref #529783
2015-11-09 18:30:13 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:30:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529783  77 57335 66613167 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:30:13 [DEBUG   ] Got command RPT from MCS: ref #529784
2015-11-09 18:30:13 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 18:30:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529784  12 57335 66613371 A  ERRORB‹ÌÍ'
2015-11-09 18:30:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:30:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:30:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:30:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:30:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:30:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:30:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:30:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:30:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:30:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:30:16 [DEBUG   ] Finished updating board flags in 1.616 seconds
2015-11-09 18:30:19 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 18:30:39 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 18:30:59 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 18:31:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:31:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:31:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:31:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:31:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:31:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:31:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:31:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:31:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:31:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:31:16 [DEBUG   ] Finished updating board flags in 1.576 seconds
2015-11-09 18:31:19 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 18:31:39 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 18:31:59 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 18:32:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:32:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:32:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:32:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:32:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:32:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:32:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:32:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:32:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:32:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:32:16 [DEBUG   ] Finished updating board flags in 1.429 seconds
2015-11-09 18:32:20 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 18:32:40 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 18:32:48 [DEBUG   ] Got command RPT from MCS: ref #529788
2015-11-09 18:32:48 [DEBUG   ] summary = ERROR
2015-11-09 18:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529788  13 57335 66768611 A  ERRORERROR'
2015-11-09 18:32:48 [DEBUG   ] Got command RPT from MCS: ref #529789
2015-11-09 18:32:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529789  77 57335 66768611 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:33:00 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 18:33:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:33:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:33:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:33:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:33:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:33:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:33:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:33:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:33:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:33:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:33:16 [DEBUG   ] Finished updating board flags in 1.379 seconds
2015-11-09 18:33:20 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 18:33:40 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 18:34:00 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 18:34:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:34:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:34:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:34:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:34:15 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:34:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:34:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:34:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:34:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:34:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:34:16 [DEBUG   ] Finished updating board flags in 1.382 seconds
2015-11-09 18:34:20 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 18:34:40 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 18:35:00 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 18:35:12 [DEBUG   ] Got command RPT from MCS: ref #529806
2015-11-09 18:35:12 [DEBUG   ] summary = ERROR
2015-11-09 18:35:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529806  13 57335 66912647 A  ERRORERROR'
2015-11-09 18:35:12 [DEBUG   ] Got command RPT from MCS: ref #529807
2015-11-09 18:35:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:35:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529807  77 57335 66912851 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:35:13 [DEBUG   ] Got command RPT from MCS: ref #529808
2015-11-09 18:35:13 [DEBUG   ] summary = ERROR
2015-11-09 18:35:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529808  13 57335 66913055 A  ERRORERROR'
2015-11-09 18:35:13 [DEBUG   ] Got command RPT from MCS: ref #529809
2015-11-09 18:35:13 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:35:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529809  77 57335 66913247 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:35:13 [DEBUG   ] Got command RPT from MCS: ref #529810
2015-11-09 18:35:13 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 18:35:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529810  12 57335 66913451 A  ERRORBŠÌÍ'
2015-11-09 18:35:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:35:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:35:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:35:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:35:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:35:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:35:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:35:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:35:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:35:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:35:17 [DEBUG   ] Finished updating board flags in 2.048 seconds
2015-11-09 18:35:20 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 18:35:40 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 18:36:00 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 18:36:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:36:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:36:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:36:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:36:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:36:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:36:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:36:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:36:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:36:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:36:17 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 18:36:17 [DEBUG   ] Finished updating board flags in 1.975 seconds
2015-11-09 18:36:20 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 18:36:40 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 18:37:00 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 18:37:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:37:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:37:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:37:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:37:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:37:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:37:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:37:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:37:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:37:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:37:17 [DEBUG   ] Finished updating board flags in 1.825 seconds
2015-11-09 18:37:20 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 18:37:40 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 18:37:48 [DEBUG   ] Got command RPT from MCS: ref #529819
2015-11-09 18:37:48 [DEBUG   ] summary = ERROR
2015-11-09 18:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529819  13 57335 67068611 A  ERRORERROR'
2015-11-09 18:37:48 [DEBUG   ] Got command RPT from MCS: ref #529820
2015-11-09 18:37:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529820  77 57335 67068611 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:38:00 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 18:38:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:38:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:38:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:38:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:38:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:38:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:38:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:38:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:38:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:38:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:38:17 [DEBUG   ] Finished updating board flags in 1.860 seconds
2015-11-09 18:38:21 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2015-11-09 18:38:41 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-09 18:39:01 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 18:39:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:39:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:39:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:39:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:39:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:39:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:39:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:39:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:39:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:39:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:39:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 18:39:17 [DEBUG   ] Finished updating board flags in 1.727 seconds
2015-11-09 18:39:21 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 18:39:41 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 18:40:01 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-09 18:40:12 [DEBUG   ] Got command RPT from MCS: ref #529839
2015-11-09 18:40:12 [DEBUG   ] summary = ERROR
2015-11-09 18:40:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529839  13 57335 67212743 A  ERRORERROR'
2015-11-09 18:40:12 [DEBUG   ] Got command RPT from MCS: ref #529840
2015-11-09 18:40:12 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:40:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529840  77 57335 67212935 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:40:13 [DEBUG   ] Got command RPT from MCS: ref #529841
2015-11-09 18:40:13 [DEBUG   ] summary = ERROR
2015-11-09 18:40:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529841  13 57335 67213139 A  ERRORERROR'
2015-11-09 18:40:13 [DEBUG   ] Got command RPT from MCS: ref #529842
2015-11-09 18:40:13 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:40:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529842  77 57335 67213343 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:40:13 [DEBUG   ] Got command RPT from MCS: ref #529843
2015-11-09 18:40:13 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 18:40:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529843  12 57335 67213547 A  ERRORBŒÌÍ'
2015-11-09 18:40:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:40:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:40:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:40:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:40:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:40:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:40:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:40:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:40:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:40:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:40:17 [DEBUG   ] Finished updating board flags in 1.615 seconds
2015-11-09 18:40:21 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 18:40:41 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 18:41:01 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 18:41:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:41:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:41:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:41:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:41:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:41:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:41:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:41:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:41:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:41:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:41:17 [DEBUG   ] Finished updating board flags in 1.588 seconds
2015-11-09 18:41:21 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 18:41:41 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 18:42:01 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 18:42:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:42:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:42:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:42:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:42:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:42:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:42:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:42:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:42:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:42:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:42:17 [DEBUG   ] Finished updating board flags in 1.468 seconds
2015-11-09 18:42:21 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 18:42:41 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 18:42:48 [DEBUG   ] Got command RPT from MCS: ref #529847
2015-11-09 18:42:48 [DEBUG   ] summary = ERROR
2015-11-09 18:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529847  13 57335 67368611 A  ERRORERROR'
2015-11-09 18:42:48 [DEBUG   ] Got command RPT from MCS: ref #529848
2015-11-09 18:42:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529848  77 57335 67368611 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:43:01 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-09 18:43:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:43:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:43:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:43:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:43:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:43:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:43:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:43:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:43:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:43:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:43:17 [DEBUG   ] Finished updating board flags in 1.468 seconds
2015-11-09 18:43:22 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2015-11-09 18:43:42 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-09 18:44:02 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 18:44:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:44:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:44:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:44:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:44:16 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:44:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:44:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:44:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:44:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:44:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:44:17 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 18:44:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 18:44:17 [DEBUG   ] Finished updating board flags in 1.483 seconds
2015-11-09 18:44:22 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2015-11-09 18:44:42 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 18:45:02 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 18:45:12 [DEBUG   ] Got command RPT from MCS: ref #529868
2015-11-09 18:45:12 [DEBUG   ] summary = ERROR
2015-11-09 18:45:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529868  13 57335 67512819 A  ERRORERROR'
2015-11-09 18:45:13 [DEBUG   ] Got command RPT from MCS: ref #529869
2015-11-09 18:45:13 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:45:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529869  77 57335 67513023 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:45:13 [DEBUG   ] Got command RPT from MCS: ref #529870
2015-11-09 18:45:13 [DEBUG   ] summary = ERROR
2015-11-09 18:45:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529870  13 57335 67513227 A  ERRORERROR'
2015-11-09 18:45:13 [DEBUG   ] Got command RPT from MCS: ref #529871
2015-11-09 18:45:13 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:45:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529871  77 57335 67513419 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:45:13 [DEBUG   ] Got command RPT from MCS: ref #529872
2015-11-09 18:45:13 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 18:45:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529872  12 57335 67513623 A  ERRORB‹ÌÍ'
2015-11-09 18:45:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:45:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:45:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:45:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:45:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:45:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:45:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:45:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:45:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:45:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:45:18 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 18:45:18 [DEBUG   ] Finished updating board flags in 2.234 seconds
2015-11-09 18:45:22 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 18:45:42 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 18:46:02 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 18:46:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:46:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:46:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:46:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:46:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:46:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:46:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:46:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:46:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:46:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:46:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 18:46:18 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 18:46:18 [DEBUG   ] Finished updating board flags in 2.154 seconds
2015-11-09 18:46:22 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 18:46:42 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 18:47:02 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 18:47:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:47:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:47:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:47:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:47:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:47:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:47:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:47:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:47:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:47:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:47:18 [DEBUG   ] Finished updating board flags in 2.032 seconds
2015-11-09 18:47:23 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 18:47:43 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 18:47:48 [DEBUG   ] Got command RPT from MCS: ref #529881
2015-11-09 18:47:48 [DEBUG   ] summary = ERROR
2015-11-09 18:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529881  13 57335 67668607 A  ERRORERROR'
2015-11-09 18:47:48 [DEBUG   ] Got command RPT from MCS: ref #529882
2015-11-09 18:47:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529882  77 57335 67668607 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:48:03 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 18:48:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:48:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:48:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:48:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:48:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:48:17 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 18:48:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:48:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:48:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:48:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:48:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:48:18 [DEBUG   ] Finished updating board flags in 1.965 seconds
2015-11-09 18:48:23 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 18:48:43 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 18:49:03 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 18:49:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:49:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:49:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:49:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:49:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:49:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:49:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:49:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:49:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:49:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:49:18 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 18:49:18 [DEBUG   ] Finished updating board flags in 1.820 seconds
2015-11-09 18:49:23 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 18:49:43 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 18:50:03 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 18:50:12 [DEBUG   ] Got command RPT from MCS: ref #529898
2015-11-09 18:50:12 [DEBUG   ] summary = ERROR
2015-11-09 18:50:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529898  13 57335 67812899 A  ERRORERROR'
2015-11-09 18:50:13 [DEBUG   ] Got command RPT from MCS: ref #529899
2015-11-09 18:50:13 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:50:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529899  77 57335 67813107 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:50:13 [DEBUG   ] Got command RPT from MCS: ref #529900
2015-11-09 18:50:13 [DEBUG   ] summary = ERROR
2015-11-09 18:50:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529900  13 57335 67813311 A  ERRORERROR'
2015-11-09 18:50:13 [DEBUG   ] Got command RPT from MCS: ref #529901
2015-11-09 18:50:13 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:50:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529901  77 57335 67813503 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:50:13 [DEBUG   ] Got command RPT from MCS: ref #529902
2015-11-09 18:50:13 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 18:50:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529902  12 57335 67813707 A  ERRORBŠÌÍ'
2015-11-09 18:50:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:50:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:50:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:50:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:50:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:50:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:50:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:50:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:50:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:50:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:50:18 [DEBUG   ] Finished updating board flags in 1.809 seconds
2015-11-09 18:50:23 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 18:50:43 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 18:51:03 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 18:51:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:51:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:51:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:51:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:51:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:51:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:51:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:51:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:51:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:51:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:51:18 [DEBUG   ] Finished updating board flags in 1.781 seconds
2015-11-09 18:51:23 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 18:51:43 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 18:52:03 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 18:52:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:52:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:52:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:52:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:52:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:52:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:52:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:52:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:52:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:52:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:52:18 [DEBUG   ] Finished updating board flags in 1.611 seconds
2015-11-09 18:52:23 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 18:52:43 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 18:52:48 [DEBUG   ] Got command RPT from MCS: ref #529906
2015-11-09 18:52:48 [DEBUG   ] summary = ERROR
2015-11-09 18:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529906  13 57335 67968615 A  ERRORERROR'
2015-11-09 18:52:48 [DEBUG   ] Got command RPT from MCS: ref #529907
2015-11-09 18:52:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529907  77 57335 67968615 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:53:03 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 18:53:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:53:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:53:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:53:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:53:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:53:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:53:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:53:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:53:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:53:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:53:18 [DEBUG   ] Finished updating board flags in 1.593 seconds
2015-11-09 18:53:24 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 18:53:44 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 18:54:04 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 18:54:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:54:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:54:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:54:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:54:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:54:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:54:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:54:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:54:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:54:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:54:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 18:54:18 [DEBUG   ] Finished updating board flags in 1.408 seconds
2015-11-09 18:54:24 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 18:54:44 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 18:55:04 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 18:55:12 [DEBUG   ] Got command RPT from MCS: ref #529927
2015-11-09 18:55:12 [DEBUG   ] summary = ERROR
2015-11-09 18:55:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529927  13 57335 68112999 A  ERRORERROR'
2015-11-09 18:55:13 [DEBUG   ] Got command RPT from MCS: ref #529928
2015-11-09 18:55:13 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:55:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529928  77 57335 68113191 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:55:13 [DEBUG   ] Got command RPT from MCS: ref #529929
2015-11-09 18:55:13 [DEBUG   ] summary = ERROR
2015-11-09 18:55:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529929  13 57335 68113395 A  ERRORERROR'
2015-11-09 18:55:13 [DEBUG   ] Got command RPT from MCS: ref #529930
2015-11-09 18:55:13 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:55:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529930  77 57335 68113599 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:55:13 [DEBUG   ] Got command RPT from MCS: ref #529931
2015-11-09 18:55:13 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 18:55:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529931  12 57335 68113791 A  ERRORB�ÌÍ'
2015-11-09 18:55:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:55:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:55:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:55:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:55:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:55:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:55:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:55:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:55:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:55:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:55:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa34, chip 1
2015-11-09 18:55:18 [DEBUG   ] Finished updating board flags in 1.429 seconds
2015-11-09 18:55:24 [DEBUG   ] Finished updating ant stats in 1.195 seconds
2015-11-09 18:55:44 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 18:56:04 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 18:56:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:56:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:56:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:56:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:56:17 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:56:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:56:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:56:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:56:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:56:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:56:18 [DEBUG   ] Finished updating board flags in 1.375 seconds
2015-11-09 18:56:24 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2015-11-09 18:56:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 18:57:04 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 18:57:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:57:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:57:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:57:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:57:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:57:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:57:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:57:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:57:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:57:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:57:19 [DEBUG   ] Finished updating board flags in 2.216 seconds
2015-11-09 18:57:24 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 18:57:44 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 18:57:48 [DEBUG   ] Got command RPT from MCS: ref #529940
2015-11-09 18:57:48 [DEBUG   ] summary = ERROR
2015-11-09 18:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529940  13 57335 68268611 A  ERRORERROR'
2015-11-09 18:57:48 [DEBUG   ] Got command RPT from MCS: ref #529941
2015-11-09 18:57:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 18:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529941  77 57335 68268611 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 18:58:04 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 18:58:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:58:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:58:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:58:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:58:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:58:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:58:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:58:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:58:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:58:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:58:19 [DEBUG   ] Finished updating board flags in 2.187 seconds
2015-11-09 18:58:25 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2015-11-09 18:58:45 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 18:59:05 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 18:59:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 18:59:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 18:59:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 18:59:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 18:59:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 18:59:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 18:59:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 18:59:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 18:59:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 18:59:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 18:59:19 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 18:59:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 18:59:19 [DEBUG   ] Finished updating board flags in 2.021 seconds
2015-11-09 18:59:25 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 18:59:45 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 19:00:05 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 19:00:13 [DEBUG   ] Got command RPT from MCS: ref #529960
2015-11-09 19:00:13 [DEBUG   ] summary = ERROR
2015-11-09 19:00:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529960  13 57335 68413079 A  ERRORERROR'
2015-11-09 19:00:13 [DEBUG   ] Got command RPT from MCS: ref #529961
2015-11-09 19:00:13 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:00:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529961  77 57335 68413283 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:00:13 [DEBUG   ] Got command RPT from MCS: ref #529962
2015-11-09 19:00:13 [DEBUG   ] summary = ERROR
2015-11-09 19:00:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529962  13 57335 68413487 A  ERRORERROR'
2015-11-09 19:00:13 [DEBUG   ] Got command RPT from MCS: ref #529963
2015-11-09 19:00:13 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:00:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529963  77 57335 68413679 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:00:13 [DEBUG   ] Got command RPT from MCS: ref #529964
2015-11-09 19:00:13 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 19:00:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529964  12 57335 68413883 A  ERRORBŒÌÍ'
2015-11-09 19:00:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:00:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:00:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:00:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:00:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:00:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:00:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:00:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:00:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:00:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:00:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2015-11-09 19:00:19 [CRITICAL] MGT underflow detected on beam(s) 1
2015-11-09 19:00:19 [DEBUG   ] Finished updating board flags in 1.983 seconds
2015-11-09 19:00:25 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 19:00:45 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 19:01:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 19:01:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:01:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:01:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:01:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:01:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:01:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:01:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:01:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:01:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:01:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:01:19 [DEBUG   ] Finished updating board flags in 1.816 seconds
2015-11-09 19:01:25 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 19:01:45 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 19:02:05 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 19:02:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:02:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:02:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:02:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:02:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:02:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:02:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:02:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:02:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:02:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:02:19 [DEBUG   ] Finished updating board flags in 1.789 seconds
2015-11-09 19:02:25 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 19:02:45 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 19:02:48 [DEBUG   ] Got command RPT from MCS: ref #529968
2015-11-09 19:02:48 [DEBUG   ] summary = ERROR
2015-11-09 19:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529968  13 57335 68568607 A  ERRORERROR'
2015-11-09 19:02:48 [DEBUG   ] Got command RPT from MCS: ref #529969
2015-11-09 19:02:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529969  77 57335 68568607 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:03:05 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 19:03:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:03:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:03:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:03:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:03:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:03:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:03:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:03:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:03:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:03:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:03:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 19:03:19 [DEBUG   ] Finished updating board flags in 1.638 seconds
2015-11-09 19:03:26 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 19:03:46 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 19:04:06 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 19:04:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:04:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:04:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:04:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:04:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:04:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa8, chip 5, bottom port
2015-11-09 19:04:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:04:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:04:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:04:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:04:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:04:19 [CRITICAL] MGT underflow detected on beam(s) 1, 3
2015-11-09 19:04:19 [DEBUG   ] Finished updating board flags in 1.564 seconds
2015-11-09 19:04:26 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 19:04:46 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 19:05:06 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 19:05:13 [DEBUG   ] Got command RPT from MCS: ref #529986
2015-11-09 19:05:13 [DEBUG   ] summary = ERROR
2015-11-09 19:05:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529986  13 57335 68713167 A  ERRORERROR'
2015-11-09 19:05:13 [DEBUG   ] Got command RPT from MCS: ref #529987
2015-11-09 19:05:13 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:05:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529987  84 57335 68713371 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:05:13 [DEBUG   ] Got command RPT from MCS: ref #529988
2015-11-09 19:05:13 [DEBUG   ] summary = ERROR
2015-11-09 19:05:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529988  13 57335 68713563 A  ERRORERROR'
2015-11-09 19:05:13 [DEBUG   ] Got command RPT from MCS: ref #529989
2015-11-09 19:05:13 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:05:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529989  84 57335 68713767 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:05:13 [DEBUG   ] Got command RPT from MCS: ref #529990
2015-11-09 19:05:13 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 19:05:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529990  12 57335 68713971 A  ERRORBŠÌÍ'
2015-11-09 19:05:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:05:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:05:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:05:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:05:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:05:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:05:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:05:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:05:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:05:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:05:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 19:05:19 [DEBUG   ] Finished updating board flags in 1.421 seconds
2015-11-09 19:05:26 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 19:05:46 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 19:06:06 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 19:06:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:06:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:06:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:06:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:06:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:06:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:06:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:06:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:06:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:06:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:06:19 [DEBUG   ] Finished updating board flags in 1.380 seconds
2015-11-09 19:06:26 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 19:06:46 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 19:07:06 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 19:07:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:07:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:07:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:07:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:07:18 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:07:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:07:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:07:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:07:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:07:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:07:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 19:07:19 [DEBUG   ] Finished updating board flags in 1.390 seconds
2015-11-09 19:07:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 19:07:46 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 19:07:48 [DEBUG   ] Got command RPT from MCS: ref #529999
2015-11-09 19:07:48 [DEBUG   ] summary = ERROR
2015-11-09 19:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   529999  13 57335 68868607 A  ERRORERROR'
2015-11-09 19:07:48 [DEBUG   ] Got command RPT from MCS: ref #530000
2015-11-09 19:07:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530000  84 57335 68868607 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:08:06 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 19:08:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:08:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:08:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:08:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:08:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:08:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:08:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:08:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:08:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:08:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:08:20 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 19:08:20 [DEBUG   ] Finished updating board flags in 2.029 seconds
2015-11-09 19:08:27 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 19:08:47 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-09 19:09:07 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 19:09:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:09:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:09:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:09:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:09:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:09:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:09:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:09:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:09:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:09:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:09:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 19:09:20 [DEBUG   ] Finished updating board flags in 1.991 seconds
2015-11-09 19:09:27 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 19:09:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 19:10:07 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 19:10:13 [DEBUG   ] Got command RPT from MCS: ref #530019
2015-11-09 19:10:13 [DEBUG   ] summary = ERROR
2015-11-09 19:10:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530019  13 57335 69013239 A  ERRORERROR'
2015-11-09 19:10:13 [DEBUG   ] Got command RPT from MCS: ref #530020
2015-11-09 19:10:13 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:10:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530020  84 57335 69013443 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:10:13 [DEBUG   ] Got command RPT from MCS: ref #530021
2015-11-09 19:10:13 [DEBUG   ] summary = ERROR
2015-11-09 19:10:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530021  13 57335 69013647 A  ERRORERROR'
2015-11-09 19:10:13 [DEBUG   ] Got command RPT from MCS: ref #530022
2015-11-09 19:10:13 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:10:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530022  84 57335 69013851 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:10:14 [DEBUG   ] Got command RPT from MCS: ref #530023
2015-11-09 19:10:14 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 19:10:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530023  12 57335 69014043 A  ERRORB‹ÌÍ'
2015-11-09 19:10:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:10:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:10:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:10:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:10:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:10:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:10:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:10:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:10:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:10:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:10:20 [DEBUG   ] Finished updating board flags in 1.815 seconds
2015-11-09 19:10:27 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 19:10:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 19:11:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 19:11:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:11:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:11:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:11:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:11:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:11:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:11:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:11:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:11:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:11:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:11:20 [DEBUG   ] Finished updating board flags in 1.809 seconds
2015-11-09 19:11:27 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 19:11:47 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 19:12:07 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 19:12:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:12:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:12:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:12:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:12:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:12:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:12:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:12:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:12:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:12:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:12:20 [DEBUG   ] Finished updating board flags in 1.789 seconds
2015-11-09 19:12:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 19:12:47 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 19:12:48 [DEBUG   ] Got command RPT from MCS: ref #530027
2015-11-09 19:12:48 [DEBUG   ] summary = ERROR
2015-11-09 19:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530027  13 57335 69168615 A  ERRORERROR'
2015-11-09 19:12:48 [DEBUG   ] Got command RPT from MCS: ref #530028
2015-11-09 19:12:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530028  84 57335 69168615 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:13:07 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 19:13:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:13:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:13:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:13:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:13:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:13:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:13:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:13:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:13:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:13:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:13:20 [DEBUG   ] Finished updating board flags in 1.614 seconds
2015-11-09 19:13:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 19:13:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 19:14:07 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 19:14:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:14:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:14:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:14:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:14:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:14:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:14:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:14:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:14:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:14:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:14:20 [DEBUG   ] Finished updating board flags in 1.603 seconds
2015-11-09 19:14:28 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-09 19:14:48 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 19:15:08 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 19:15:13 [DEBUG   ] Got command RPT from MCS: ref #530052
2015-11-09 19:15:13 [DEBUG   ] summary = ERROR
2015-11-09 19:15:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530052  13 57335 69313323 A  ERRORERROR'
2015-11-09 19:15:13 [DEBUG   ] Got command RPT from MCS: ref #530053
2015-11-09 19:15:13 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:15:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530053  84 57335 69313527 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:15:13 [DEBUG   ] Got command RPT from MCS: ref #530054
2015-11-09 19:15:13 [DEBUG   ] summary = ERROR
2015-11-09 19:15:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530054  13 57335 69313731 A  ERRORERROR'
2015-11-09 19:15:13 [DEBUG   ] Got command RPT from MCS: ref #530055
2015-11-09 19:15:13 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:15:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530055  84 57335 69313935 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:15:14 [DEBUG   ] Got command RPT from MCS: ref #530056
2015-11-09 19:15:14 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 19:15:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530056  12 57335 69314127 A  ERRORBŒÌÍ'
2015-11-09 19:15:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:15:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:15:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:15:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:15:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:15:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:15:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:15:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:15:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:15:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:15:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 19:15:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 19:15:20 [DEBUG   ] Finished updating board flags in 1.590 seconds
2015-11-09 19:15:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 19:15:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 19:16:08 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 19:16:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:16:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:16:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:16:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:16:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:16:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:16:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:16:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:16:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:16:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:16:20 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 19:16:20 [DEBUG   ] Finished updating board flags in 1.460 seconds
2015-11-09 19:16:28 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 19:16:48 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 19:17:08 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 19:17:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:17:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:17:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:17:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:17:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:17:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:17:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:17:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:17:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:17:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:17:20 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 19:17:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 19:17:20 [DEBUG   ] Finished updating board flags in 1.452 seconds
2015-11-09 19:17:28 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 19:17:48 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 19:17:48 [DEBUG   ] Got command RPT from MCS: ref #530066
2015-11-09 19:17:48 [DEBUG   ] summary = ERROR
2015-11-09 19:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530066  13 57335 69468615 A  ERRORERROR'
2015-11-09 19:17:48 [DEBUG   ] Got command RPT from MCS: ref #530067
2015-11-09 19:17:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530067  84 57335 69468615 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:18:08 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 19:18:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:18:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:18:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:18:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:18:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:18:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:18:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:18:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:18:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:18:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:18:20 [DEBUG   ] Finished updating board flags in 1.481 seconds
2015-11-09 19:18:28 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 19:18:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 19:19:08 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 19:19:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:19:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:19:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:19:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:19:19 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:19:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:19:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:19:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:19:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:19:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:19:20 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 19:19:20 [DEBUG   ] Finished updating board flags in 1.479 seconds
2015-11-09 19:19:29 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 19:19:49 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 19:20:09 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 19:20:13 [DEBUG   ] Got command RPT from MCS: ref #530083
2015-11-09 19:20:13 [DEBUG   ] summary = ERROR
2015-11-09 19:20:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530083  13 57335 69613415 A  ERRORERROR'
2015-11-09 19:20:13 [DEBUG   ] Got command RPT from MCS: ref #530084
2015-11-09 19:20:13 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:20:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530084  84 57335 69613619 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:20:13 [DEBUG   ] Got command RPT from MCS: ref #530085
2015-11-09 19:20:13 [DEBUG   ] summary = ERROR
2015-11-09 19:20:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530085  13 57335 69613811 A  ERRORERROR'
2015-11-09 19:20:14 [DEBUG   ] Got command RPT from MCS: ref #530086
2015-11-09 19:20:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:20:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530086  84 57335 69614015 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:20:14 [DEBUG   ] Got command RPT from MCS: ref #530087
2015-11-09 19:20:14 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 19:20:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530087  12 57335 69614219 A  ERRORBŠÌÍ'
2015-11-09 19:20:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:20:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:20:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:20:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:20:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:20:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:20:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:20:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:20:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:20:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:20:21 [DEBUG   ] Finished updating board flags in 2.209 seconds
2015-11-09 19:20:29 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 19:20:49 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 19:21:09 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 19:21:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:21:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:21:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:21:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:21:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:21:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:21:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:21:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:21:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:21:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:21:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 19:21:21 [DEBUG   ] Finished updating board flags in 2.128 seconds
2015-11-09 19:21:29 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 19:21:49 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2015-11-09 19:22:09 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 19:22:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:22:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:22:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:22:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:22:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:22:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 19:22:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:22:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:22:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:22:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:22:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:22:21 [DEBUG   ] Finished updating board flags in 2.012 seconds
2015-11-09 19:22:29 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 19:22:48 [DEBUG   ] Got command RPT from MCS: ref #530091
2015-11-09 19:22:48 [DEBUG   ] summary = ERROR
2015-11-09 19:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530091  13 57335 69768619 A  ERRORERROR'
2015-11-09 19:22:48 [DEBUG   ] Got command RPT from MCS: ref #530092
2015-11-09 19:22:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530092  84 57335 69768619 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:22:49 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 19:23:09 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 19:23:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:23:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:23:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:23:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:23:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:23:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 19:23:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:23:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:23:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:23:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:23:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:23:21 [DEBUG   ] Finished updating board flags in 1.983 seconds
2015-11-09 19:23:29 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 19:23:49 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 19:24:09 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 19:24:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:24:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:24:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:24:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:24:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:24:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:24:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:24:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:24:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:24:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:24:21 [DEBUG   ] Finished updating board flags in 1.865 seconds
2015-11-09 19:24:29 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 19:24:49 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 19:25:09 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 19:25:13 [DEBUG   ] Got command RPT from MCS: ref #530112
2015-11-09 19:25:13 [DEBUG   ] summary = ERROR
2015-11-09 19:25:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530112  13 57335 69913495 A  ERRORERROR'
2015-11-09 19:25:13 [DEBUG   ] Got command RPT from MCS: ref #530113
2015-11-09 19:25:13 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:25:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530113  84 57335 69913699 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:25:13 [DEBUG   ] Got command RPT from MCS: ref #530114
2015-11-09 19:25:13 [DEBUG   ] summary = ERROR
2015-11-09 19:25:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530114  13 57335 69913903 A  ERRORERROR'
2015-11-09 19:25:14 [DEBUG   ] Got command RPT from MCS: ref #530115
2015-11-09 19:25:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:25:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530115  84 57335 69914095 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:25:14 [DEBUG   ] Got command RPT from MCS: ref #530116
2015-11-09 19:25:14 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 19:25:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530116  12 57335 69914299 A  ERRORBŠÌÍ'
2015-11-09 19:25:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:25:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:25:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:25:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:25:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:25:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:25:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:25:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:25:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:25:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:25:21 [DEBUG   ] Finished updating board flags in 1.748 seconds
2015-11-09 19:25:29 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 19:25:50 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-09 19:26:10 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 19:26:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:26:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:26:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:26:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:26:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:26:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:26:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:26:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:26:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:26:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:26:21 [DEBUG   ] Finished updating board flags in 1.613 seconds
2015-11-09 19:26:30 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 19:26:50 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 19:27:10 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 19:27:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:27:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:27:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:27:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:27:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:27:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:27:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:27:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:27:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:27:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:27:21 [DEBUG   ] Finished updating board flags in 1.589 seconds
2015-11-09 19:27:30 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 19:27:48 [DEBUG   ] Got command RPT from MCS: ref #530125
2015-11-09 19:27:48 [DEBUG   ] summary = ERROR
2015-11-09 19:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530125  13 57335 70068611 A  ERRORERROR'
2015-11-09 19:27:48 [DEBUG   ] Got command RPT from MCS: ref #530126
2015-11-09 19:27:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530126  84 57335 70068611 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:27:50 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 19:28:10 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 19:28:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:28:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:28:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:28:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:28:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:28:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:28:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:28:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:28:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:28:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:28:21 [DEBUG   ] Finished updating board flags in 1.414 seconds
2015-11-09 19:28:30 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 19:28:50 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 19:29:10 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 19:29:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:29:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:29:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:29:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:29:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:29:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:29:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:29:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:29:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:29:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:29:21 [DEBUG   ] Finished updating board flags in 1.410 seconds
2015-11-09 19:29:30 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 19:29:50 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 19:30:10 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 19:30:13 [DEBUG   ] Got command RPT from MCS: ref #530145
2015-11-09 19:30:13 [DEBUG   ] summary = ERROR
2015-11-09 19:30:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530145  13 57335 70213599 A  ERRORERROR'
2015-11-09 19:30:13 [DEBUG   ] Got command RPT from MCS: ref #530146
2015-11-09 19:30:13 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:30:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530146  84 57335 70213791 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:30:13 [DEBUG   ] Got command RPT from MCS: ref #530147
2015-11-09 19:30:13 [DEBUG   ] summary = ERROR
2015-11-09 19:30:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530147  13 57335 70213995 A  ERRORERROR'
2015-11-09 19:30:14 [DEBUG   ] Got command RPT from MCS: ref #530148
2015-11-09 19:30:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:30:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530148  84 57335 70214199 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:30:14 [DEBUG   ] Got command RPT from MCS: ref #530149
2015-11-09 19:30:14 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 19:30:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530149  12 57335 70214391 A  ERRORB�ÌÍ'
2015-11-09 19:30:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:30:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:30:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:30:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:30:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:30:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:30:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:30:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:30:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:30:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:30:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 19:30:21 [DEBUG   ] Finished updating board flags in 1.413 seconds
2015-11-09 19:30:30 [DEBUG   ] Finished updating ant stats in 0.595 seconds
2015-11-09 19:30:51 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 19:31:11 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 19:31:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:31:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:31:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:31:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:31:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:31:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:31:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:31:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:31:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:31:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:31:21 [DEBUG   ] Finished updating board flags in 1.407 seconds
2015-11-09 19:31:31 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 19:31:51 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 19:32:11 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 19:32:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:32:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:32:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:32:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:32:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:32:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:32:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:32:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:32:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:32:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:32:21 [DEBUG   ] Finished updating board flags in 1.406 seconds
2015-11-09 19:32:31 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 19:32:48 [DEBUG   ] Got command RPT from MCS: ref #530153
2015-11-09 19:32:48 [DEBUG   ] summary = ERROR
2015-11-09 19:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530153  13 57335 70368619 A  ERRORERROR'
2015-11-09 19:32:48 [DEBUG   ] Got command RPT from MCS: ref #530154
2015-11-09 19:32:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530154  84 57335 70368619 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:32:51 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 19:33:11 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 19:33:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:33:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:33:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:33:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:33:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:33:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:33:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:33:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:33:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:33:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:33:21 [DEBUG   ] Finished updating board flags in 1.356 seconds
2015-11-09 19:33:31 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 19:33:51 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 19:34:11 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 19:34:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:34:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:34:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:34:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:34:20 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:34:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:34:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:34:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:34:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:34:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:34:21 [DEBUG   ] Finished updating board flags in 1.374 seconds
2015-11-09 19:34:31 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 19:34:51 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 19:35:11 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 19:35:13 [DEBUG   ] Got command RPT from MCS: ref #530171
2015-11-09 19:35:13 [DEBUG   ] summary = ERROR
2015-11-09 19:35:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530171  13 57335 70513687 A  ERRORERROR'
2015-11-09 19:35:13 [DEBUG   ] Got command RPT from MCS: ref #530172
2015-11-09 19:35:13 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:35:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530172  84 57335 70513891 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:35:14 [DEBUG   ] Got command RPT from MCS: ref #530173
2015-11-09 19:35:14 [DEBUG   ] summary = ERROR
2015-11-09 19:35:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530173  13 57335 70514095 A  ERRORERROR'
2015-11-09 19:35:14 [DEBUG   ] Got command RPT from MCS: ref #530174
2015-11-09 19:35:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:35:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530174  84 57335 70514287 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:35:14 [DEBUG   ] Got command RPT from MCS: ref #530175
2015-11-09 19:35:14 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 19:35:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530175  12 57335 70514491 A  ERRORB‹ÌÍ'
2015-11-09 19:35:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:35:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:35:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:35:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:35:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:35:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:35:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:35:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:35:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:35:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:35:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 19:35:22 [DEBUG   ] Finished updating board flags in 2.050 seconds
2015-11-09 19:35:31 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 19:35:51 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 19:36:11 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 19:36:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:36:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:36:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:36:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:36:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:36:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:36:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:36:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:36:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:36:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:36:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 19:36:22 [DEBUG   ] Finished updating board flags in 1.987 seconds
2015-11-09 19:36:31 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 19:36:51 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 19:37:12 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 19:37:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:37:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:37:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:37:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:37:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:37:21 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 19:37:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:37:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:37:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:37:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:37:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:37:22 [DEBUG   ] Finished updating board flags in 1.815 seconds
2015-11-09 19:37:32 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 19:37:48 [DEBUG   ] Got command RPT from MCS: ref #530184
2015-11-09 19:37:48 [DEBUG   ] summary = ERROR
2015-11-09 19:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530184  13 57335 70668615 A  ERRORERROR'
2015-11-09 19:37:48 [DEBUG   ] Got command RPT from MCS: ref #530185
2015-11-09 19:37:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530185  84 57335 70668615 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:37:52 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 19:38:12 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 19:38:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:38:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:38:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:38:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:38:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:38:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:38:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:38:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:38:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:38:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:38:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 19:38:22 [DEBUG   ] Finished updating board flags in 1.793 seconds
2015-11-09 19:38:32 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 19:38:52 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 19:39:12 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2015-11-09 19:39:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:39:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:39:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:39:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:39:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:39:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:39:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:39:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:39:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:39:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:39:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 19:39:22 [DEBUG   ] Finished updating board flags in 1.607 seconds
2015-11-09 19:39:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 19:39:52 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 19:40:12 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 19:40:13 [DEBUG   ] Got command RPT from MCS: ref #530204
2015-11-09 19:40:13 [DEBUG   ] summary = ERROR
2015-11-09 19:40:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530204  13 57335 70813779 A  ERRORERROR'
2015-11-09 19:40:13 [DEBUG   ] Got command RPT from MCS: ref #530205
2015-11-09 19:40:13 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:40:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530205  84 57335 70813971 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:40:14 [DEBUG   ] Got command RPT from MCS: ref #530206
2015-11-09 19:40:14 [DEBUG   ] summary = ERROR
2015-11-09 19:40:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530206  13 57335 70814175 A  ERRORERROR'
2015-11-09 19:40:14 [DEBUG   ] Got command RPT from MCS: ref #530207
2015-11-09 19:40:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:40:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530207  84 57335 70814379 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:40:14 [DEBUG   ] Got command RPT from MCS: ref #530208
2015-11-09 19:40:14 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 19:40:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530208  12 57335 70814583 A  ERRORBŠÌÍ'
2015-11-09 19:40:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:40:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:40:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:40:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:40:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:40:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:40:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:40:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:40:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:40:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:40:22 [DEBUG   ] Finished updating board flags in 1.595 seconds
2015-11-09 19:40:32 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 19:40:52 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 19:41:12 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 19:41:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:41:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:41:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:41:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:41:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:41:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:41:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:41:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:41:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:41:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:41:22 [DEBUG   ] Finished updating board flags in 1.415 seconds
2015-11-09 19:41:32 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 19:41:52 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 19:42:13 [DEBUG   ] Finished updating ant stats in 1.800 seconds
2015-11-09 19:42:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:42:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:42:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:42:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:42:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:42:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:42:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:42:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:42:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:42:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:42:22 [DEBUG   ] Finished updating board flags in 1.383 seconds
2015-11-09 19:42:32 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 19:42:48 [DEBUG   ] Got command RPT from MCS: ref #530212
2015-11-09 19:42:48 [DEBUG   ] summary = ERROR
2015-11-09 19:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530212  13 57335 70968619 A  ERRORERROR'
2015-11-09 19:42:48 [DEBUG   ] Got command RPT from MCS: ref #530213
2015-11-09 19:42:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530213  84 57335 70968619 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:42:52 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 19:43:12 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 19:43:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:43:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:43:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:43:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:43:21 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:43:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:43:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:43:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:43:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:43:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:43:23 [DEBUG   ] Finished updating board flags in 1.595 seconds
2015-11-09 19:43:32 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2015-11-09 19:43:52 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 19:44:12 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 19:44:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:44:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:44:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:44:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:44:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:44:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:44:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:44:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:44:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:44:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:44:23 [DEBUG   ] Finished updating board flags in 2.146 seconds
2015-11-09 19:44:33 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-09 19:44:53 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 19:45:13 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 19:45:13 [DEBUG   ] Got command RPT from MCS: ref #530233
2015-11-09 19:45:13 [DEBUG   ] summary = ERROR
2015-11-09 19:45:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530233  13 57335 71113875 A  ERRORERROR'
2015-11-09 19:45:14 [DEBUG   ] Got command RPT from MCS: ref #530234
2015-11-09 19:45:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:45:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530234  84 57335 71114067 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:45:14 [DEBUG   ] Got command RPT from MCS: ref #530235
2015-11-09 19:45:14 [DEBUG   ] summary = ERROR
2015-11-09 19:45:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530235  13 57335 71114271 A  ERRORERROR'
2015-11-09 19:45:14 [DEBUG   ] Got command RPT from MCS: ref #530236
2015-11-09 19:45:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:45:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530236  84 57335 71114475 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:45:14 [DEBUG   ] Got command RPT from MCS: ref #530237
2015-11-09 19:45:14 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 19:45:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530237  12 57335 71114679 A  ERRORBŒÌÍ'
2015-11-09 19:45:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:45:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:45:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:45:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:45:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:45:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:45:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:45:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:45:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:45:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:45:23 [DEBUG   ] Finished updating board flags in 2.017 seconds
2015-11-09 19:45:33 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 19:45:53 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 19:46:13 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 19:46:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:46:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:46:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:46:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:46:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:46:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:46:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:46:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:46:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:46:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:46:23 [DEBUG   ] Finished updating board flags in 1.860 seconds
2015-11-09 19:46:33 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 19:46:53 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-09 19:47:13 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 19:47:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:47:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:47:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:47:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:47:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:47:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:47:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:47:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:47:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:47:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:47:23 [DEBUG   ] Finished updating board flags in 1.818 seconds
2015-11-09 19:47:33 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 19:47:48 [DEBUG   ] Got command RPT from MCS: ref #530246
2015-11-09 19:47:48 [DEBUG   ] summary = ERROR
2015-11-09 19:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530246  13 57335 71268615 A  ERRORERROR'
2015-11-09 19:47:48 [DEBUG   ] Got command RPT from MCS: ref #530247
2015-11-09 19:47:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530247  84 57335 71268615 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:47:53 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 19:48:13 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 19:48:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:48:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:48:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:48:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:48:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:48:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:48:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:48:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:48:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:48:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:48:23 [DEBUG   ] Finished updating board flags in 1.791 seconds
2015-11-09 19:48:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 19:48:53 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 19:49:13 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 19:49:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:49:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:49:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:49:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:49:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:49:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:49:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:49:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:49:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:49:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:49:23 [DEBUG   ] Finished updating board flags in 1.603 seconds
2015-11-09 19:49:34 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-09 19:49:54 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 19:50:13 [DEBUG   ] Got command RPT from MCS: ref #530263
2015-11-09 19:50:13 [DEBUG   ] summary = ERROR
2015-11-09 19:50:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530263  13 57335 71413971 A  ERRORERROR'
2015-11-09 19:50:14 [DEBUG   ] Got command RPT from MCS: ref #530264
2015-11-09 19:50:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:50:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530264  84 57335 71414175 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:50:14 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 19:50:14 [DEBUG   ] Got command RPT from MCS: ref #530265
2015-11-09 19:50:14 [DEBUG   ] summary = ERROR
2015-11-09 19:50:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530265  13 57335 71414367 A  ERRORERROR'
2015-11-09 19:50:14 [DEBUG   ] Got command RPT from MCS: ref #530266
2015-11-09 19:50:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:50:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530266  84 57335 71414571 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:50:14 [DEBUG   ] Got command RPT from MCS: ref #530267
2015-11-09 19:50:14 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 19:50:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530267  12 57335 71414775 A  ERRORBŒÌÍ'
2015-11-09 19:50:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:50:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:50:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:50:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:50:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:50:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:50:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:50:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:50:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:50:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:50:23 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 19:50:23 [DEBUG   ] Finished updating board flags in 1.613 seconds
2015-11-09 19:50:34 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 19:50:54 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 19:51:14 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 19:51:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:51:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:51:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:51:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:51:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:51:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:51:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:51:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:51:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:51:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:51:23 [DEBUG   ] Finished updating board flags in 1.579 seconds
2015-11-09 19:51:34 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 19:51:54 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 19:52:14 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 19:52:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:52:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:52:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:52:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:52:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:52:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:52:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:52:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:52:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:52:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:52:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 19:52:23 [DEBUG   ] Finished updating board flags in 1.414 seconds
2015-11-09 19:52:34 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 19:52:48 [DEBUG   ] Got command RPT from MCS: ref #530271
2015-11-09 19:52:48 [DEBUG   ] summary = ERROR
2015-11-09 19:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530271  13 57335 71568627 A  ERRORERROR'
2015-11-09 19:52:48 [DEBUG   ] Got command RPT from MCS: ref #530272
2015-11-09 19:52:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530272  84 57335 71568627 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:52:54 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 19:53:14 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-09 19:53:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:53:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:53:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:53:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:53:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:53:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:53:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:53:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:53:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:53:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:53:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 19:53:23 [DEBUG   ] Finished updating board flags in 1.408 seconds
2015-11-09 19:53:34 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 19:53:54 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 19:54:14 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 19:54:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:54:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:54:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:54:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:54:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:54:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:54:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:54:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:54:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:54:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:54:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 19:54:23 [DEBUG   ] Finished updating board flags in 1.458 seconds
2015-11-09 19:54:34 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 19:54:54 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 19:55:14 [DEBUG   ] Got command RPT from MCS: ref #530292
2015-11-09 19:55:14 [DEBUG   ] summary = ERROR
2015-11-09 19:55:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530292  13 57335 71714067 A  ERRORERROR'
2015-11-09 19:55:14 [DEBUG   ] Got command RPT from MCS: ref #530293
2015-11-09 19:55:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:55:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530293  84 57335 71714271 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:55:14 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 19:55:14 [DEBUG   ] Got command RPT from MCS: ref #530294
2015-11-09 19:55:14 [DEBUG   ] summary = ERROR
2015-11-09 19:55:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530294  13 57335 71714463 A  ERRORERROR'
2015-11-09 19:55:14 [DEBUG   ] Got command RPT from MCS: ref #530295
2015-11-09 19:55:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:55:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530295  84 57335 71714667 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:55:14 [DEBUG   ] Got command RPT from MCS: ref #530296
2015-11-09 19:55:14 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 19:55:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530296  12 57335 71714871 A  ERRORB‹ÌÍ'
2015-11-09 19:55:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:55:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:55:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:55:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:55:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:55:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:55:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:55:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:55:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:55:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:55:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 19:55:23 [DEBUG   ] Finished updating board flags in 1.461 seconds
2015-11-09 19:55:34 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 19:55:55 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 19:56:15 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 19:56:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:56:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:56:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:56:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:56:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:56:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:56:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:56:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:56:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:56:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:56:23 [DEBUG   ] Finished updating board flags in 1.448 seconds
2015-11-09 19:56:35 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 19:56:55 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 19:57:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 19:57:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:57:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:57:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:57:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:57:22 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:57:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 19:57:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:57:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:57:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:57:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:57:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:57:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 19:57:23 [DEBUG   ] Finished updating board flags in 1.453 seconds
2015-11-09 19:57:35 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 19:57:48 [DEBUG   ] Got command RPT from MCS: ref #530305
2015-11-09 19:57:48 [DEBUG   ] summary = ERROR
2015-11-09 19:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530305  13 57335 71868619 A  ERRORERROR'
2015-11-09 19:57:48 [DEBUG   ] Got command RPT from MCS: ref #530306
2015-11-09 19:57:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 19:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530306  84 57335 71868619 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 19:57:55 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-09 19:58:15 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 19:58:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:58:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:58:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:58:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:58:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:58:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:58:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:58:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:58:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:58:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:58:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 19:58:24 [DEBUG   ] Finished updating board flags in 2.236 seconds
2015-11-09 19:58:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 19:58:55 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 19:59:15 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 19:59:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 19:59:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 19:59:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 19:59:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 19:59:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 19:59:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 19:59:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 19:59:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 19:59:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 19:59:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 19:59:24 [DEBUG   ] Finished updating board flags in 2.138 seconds
2015-11-09 19:59:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 19:59:55 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 20:00:14 [DEBUG   ] Got command RPT from MCS: ref #530325
2015-11-09 20:00:14 [DEBUG   ] summary = ERROR
2015-11-09 20:00:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530325  13 57335 72014155 A  ERRORERROR'
2015-11-09 20:00:14 [DEBUG   ] Got command RPT from MCS: ref #530326
2015-11-09 20:00:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:00:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530326  84 57335 72014359 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:00:14 [DEBUG   ] Got command RPT from MCS: ref #530327
2015-11-09 20:00:14 [DEBUG   ] summary = ERROR
2015-11-09 20:00:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530327  13 57335 72014564 A  ERRORERROR'
2015-11-09 20:00:14 [DEBUG   ] Got command RPT from MCS: ref #530328
2015-11-09 20:00:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:00:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530328  84 57335 72014767 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:00:14 [DEBUG   ] Got command RPT from MCS: ref #530329
2015-11-09 20:00:14 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 20:00:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530329  12 57335 72014959 A  ERRORB‹ÌÍ'
2015-11-09 20:00:15 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 20:00:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:00:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:00:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:00:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:00:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:00:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:00:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:00:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:00:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:00:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:00:24 [DEBUG   ] Finished updating board flags in 2.010 seconds
2015-11-09 20:00:35 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 20:00:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 20:01:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 20:01:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:01:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:01:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:01:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:01:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:01:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:01:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:01:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:01:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:01:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:01:24 [DEBUG   ] Finished updating board flags in 1.996 seconds
2015-11-09 20:01:35 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 20:01:56 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-09 20:02:16 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 20:02:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:02:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:02:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:02:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:02:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:02:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:02:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:02:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:02:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:02:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:02:24 [DEBUG   ] Finished updating board flags in 1.802 seconds
2015-11-09 20:02:36 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 20:02:48 [DEBUG   ] Got command RPT from MCS: ref #530333
2015-11-09 20:02:48 [DEBUG   ] summary = ERROR
2015-11-09 20:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530333  13 57335 72168627 A  ERRORERROR'
2015-11-09 20:02:48 [DEBUG   ] Got command RPT from MCS: ref #530334
2015-11-09 20:02:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530334  84 57335 72168627 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:02:56 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 20:03:16 [DEBUG   ] Finished updating ant stats in 1.190 seconds
2015-11-09 20:03:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:03:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:03:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:03:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:03:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:03:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:03:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:03:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:03:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:03:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:03:24 [DEBUG   ] Finished updating board flags in 1.814 seconds
2015-11-09 20:03:36 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 20:03:56 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 20:04:16 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 20:04:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:04:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:04:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:04:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:04:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:04:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:04:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:04:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:04:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:04:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:04:24 [DEBUG   ] Finished updating board flags in 1.784 seconds
2015-11-09 20:04:36 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-09 20:04:56 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 20:05:14 [DEBUG   ] Got command RPT from MCS: ref #530351
2015-11-09 20:05:14 [DEBUG   ] summary = ERROR
2015-11-09 20:05:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530351  13 57335 72314235 A  ERRORERROR'
2015-11-09 20:05:14 [DEBUG   ] Got command RPT from MCS: ref #530352
2015-11-09 20:05:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:05:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530352  84 57335 72314439 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:05:14 [DEBUG   ] Got command RPT from MCS: ref #530353
2015-11-09 20:05:14 [DEBUG   ] summary = ERROR
2015-11-09 20:05:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530353  13 57335 72314643 A  ERRORERROR'
2015-11-09 20:05:14 [DEBUG   ] Got command RPT from MCS: ref #530354
2015-11-09 20:05:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:05:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530354  84 57335 72314847 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:05:15 [DEBUG   ] Got command RPT from MCS: ref #530355
2015-11-09 20:05:15 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 20:05:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530355  12 57335 72315039 A  ERRORBŒÌÍ'
2015-11-09 20:05:16 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 20:05:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:05:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:05:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:05:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:05:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:05:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:05:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:05:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:05:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:05:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:05:24 [DEBUG   ] Finished updating board flags in 1.636 seconds
2015-11-09 20:05:36 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 20:05:56 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 20:06:16 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 20:06:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:06:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:06:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:06:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:06:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:06:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:06:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:06:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:06:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:06:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:06:24 [DEBUG   ] Finished updating board flags in 1.565 seconds
2015-11-09 20:06:36 [DEBUG   ] Finished updating ant stats in 0.595 seconds
2015-11-09 20:06:57 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-09 20:07:17 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 20:07:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:07:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:07:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:07:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:07:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:07:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:07:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:07:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:07:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:07:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:07:24 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 20:07:24 [DEBUG   ] Finished updating board flags in 1.422 seconds
2015-11-09 20:07:37 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 20:07:48 [DEBUG   ] Got command RPT from MCS: ref #530364
2015-11-09 20:07:48 [DEBUG   ] summary = ERROR
2015-11-09 20:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530364  13 57335 72468627 A  ERRORERROR'
2015-11-09 20:07:48 [DEBUG   ] Got command RPT from MCS: ref #530365
2015-11-09 20:07:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530365  84 57335 72468627 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:07:57 [DEBUG   ] Finished updating ant stats in 1.210 seconds
2015-11-09 20:08:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 20:08:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:08:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:08:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:08:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:08:23 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:08:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:08:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:08:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:08:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:08:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:08:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 20:08:24 [DEBUG   ] Finished updating board flags in 1.388 seconds
2015-11-09 20:08:37 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 20:08:57 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 20:09:17 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 20:09:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:09:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:09:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:09:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:09:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:09:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:09:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:09:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:09:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:09:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:09:25 [DEBUG   ] Finished updating board flags in 2.209 seconds
2015-11-09 20:09:37 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 20:09:57 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 20:10:14 [DEBUG   ] Got command RPT from MCS: ref #530384
2015-11-09 20:10:14 [DEBUG   ] summary = ERROR
2015-11-09 20:10:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530384  13 57335 72614331 A  ERRORERROR'
2015-11-09 20:10:14 [DEBUG   ] Got command RPT from MCS: ref #530385
2015-11-09 20:10:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:10:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530385  84 57335 72614535 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:10:14 [DEBUG   ] Got command RPT from MCS: ref #530386
2015-11-09 20:10:14 [DEBUG   ] summary = ERROR
2015-11-09 20:10:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530386  13 57335 72614739 A  ERRORERROR'
2015-11-09 20:10:14 [DEBUG   ] Got command RPT from MCS: ref #530387
2015-11-09 20:10:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:10:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530387  84 57335 72614931 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:10:15 [DEBUG   ] Got command RPT from MCS: ref #530388
2015-11-09 20:10:15 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 20:10:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530388  12 57335 72615139 A  ERRORBŒÌÍ'
2015-11-09 20:10:17 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 20:10:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:10:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:10:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:10:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:10:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:10:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:10:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:10:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:10:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:10:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:10:25 [DEBUG   ] Finished updating board flags in 2.212 seconds
2015-11-09 20:10:37 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 20:10:57 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 20:11:17 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 20:11:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:11:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:11:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:11:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:11:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:11:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:11:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:11:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:11:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:11:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:11:25 [DEBUG   ] Finished updating board flags in 2.191 seconds
2015-11-09 20:11:37 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 20:11:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 20:12:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 20:12:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:12:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:12:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:12:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:12:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:12:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:12:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:12:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:12:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:12:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:12:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 20:12:25 [DEBUG   ] Finished updating board flags in 2.015 seconds
2015-11-09 20:12:37 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 20:12:48 [DEBUG   ] Got command RPT from MCS: ref #530392
2015-11-09 20:12:48 [DEBUG   ] summary = ERROR
2015-11-09 20:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530392  13 57335 72768627 A  ERRORERROR'
2015-11-09 20:12:48 [DEBUG   ] Got command RPT from MCS: ref #530393
2015-11-09 20:12:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530393  84 57335 72768627 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:12:57 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 20:13:17 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 20:13:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:13:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:13:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:13:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:13:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:13:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:13:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:13:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:13:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:13:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:13:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 20:13:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 20:13:25 [DEBUG   ] Finished updating board flags in 1.985 seconds
2015-11-09 20:13:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 20:13:57 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 20:14:17 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 20:14:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:14:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:14:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:14:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:14:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:14:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:14:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:14:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:14:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:14:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:14:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 20:14:25 [DEBUG   ] Finished updating board flags in 1.816 seconds
2015-11-09 20:14:38 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-09 20:14:58 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 20:15:14 [DEBUG   ] Got command RPT from MCS: ref #530413
2015-11-09 20:15:14 [DEBUG   ] summary = ERROR
2015-11-09 20:15:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530413  13 57335 72914423 A  ERRORERROR'
2015-11-09 20:15:14 [DEBUG   ] Got command RPT from MCS: ref #530414
2015-11-09 20:15:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:15:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530414  84 57335 72914615 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:15:14 [DEBUG   ] Got command RPT from MCS: ref #530415
2015-11-09 20:15:14 [DEBUG   ] summary = ERROR
2015-11-09 20:15:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530415  13 57335 72914819 A  ERRORERROR'
2015-11-09 20:15:15 [DEBUG   ] Got command RPT from MCS: ref #530416
2015-11-09 20:15:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:15:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530416  84 57335 72915023 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:15:15 [DEBUG   ] Got command RPT from MCS: ref #530417
2015-11-09 20:15:15 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 20:15:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530417  12 57335 72915227 A  ERRORBŠÌÍ'
2015-11-09 20:15:18 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 20:15:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:15:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:15:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:15:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:15:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:15:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:15:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:15:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:15:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:15:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:15:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 20:15:25 [DEBUG   ] Finished updating board flags in 1.804 seconds
2015-11-09 20:15:38 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 20:15:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 20:16:18 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 20:16:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:16:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:16:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:16:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:16:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:16:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:16:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:16:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:16:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:16:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:16:25 [DEBUG   ] Finished updating board flags in 1.792 seconds
2015-11-09 20:16:38 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 20:16:58 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 20:17:18 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 20:17:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:17:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:17:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:17:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:17:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:17:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:17:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:17:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:17:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:17:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:17:25 [DEBUG   ] Finished updating board flags in 1.609 seconds
2015-11-09 20:17:38 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 20:17:48 [DEBUG   ] Got command RPT from MCS: ref #530426
2015-11-09 20:17:48 [DEBUG   ] summary = ERROR
2015-11-09 20:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530426  13 57335 73068631 A  ERRORERROR'
2015-11-09 20:17:48 [DEBUG   ] Got command RPT from MCS: ref #530427
2015-11-09 20:17:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530427  84 57335 73068631 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:17:58 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 20:18:18 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 20:18:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:18:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:18:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:18:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:18:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:18:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:18:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:18:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:18:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:18:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:18:25 [DEBUG   ] Finished updating board flags in 1.612 seconds
2015-11-09 20:18:38 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 20:18:58 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 20:19:18 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 20:19:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:19:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:19:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:19:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:19:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:19:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:19:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:19:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:19:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:19:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:19:25 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 20:19:25 [DEBUG   ] Finished updating board flags in 1.699 seconds
2015-11-09 20:19:39 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 20:19:59 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 20:20:14 [DEBUG   ] Got command RPT from MCS: ref #530443
2015-11-09 20:20:14 [DEBUG   ] summary = ERROR
2015-11-09 20:20:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530443  13 57335 73214511 A  ERRORERROR'
2015-11-09 20:20:14 [DEBUG   ] Got command RPT from MCS: ref #530444
2015-11-09 20:20:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:20:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530444  84 57335 73214715 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:20:14 [DEBUG   ] Got command RPT from MCS: ref #530445
2015-11-09 20:20:14 [DEBUG   ] summary = ERROR
2015-11-09 20:20:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530445  13 57335 73214919 A  ERRORERROR'
2015-11-09 20:20:15 [DEBUG   ] Got command RPT from MCS: ref #530446
2015-11-09 20:20:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:20:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530446  84 57335 73215111 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:20:15 [DEBUG   ] Got command RPT from MCS: ref #530447
2015-11-09 20:20:15 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 20:20:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530447  12 57335 73215315 A  ERRORBŒÌÍ'
2015-11-09 20:20:19 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 20:20:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:20:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:20:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:20:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:20:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:20:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:20:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:20:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:20:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:20:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:20:25 [DEBUG   ] Finished updating board flags in 1.486 seconds
2015-11-09 20:20:39 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 20:20:59 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 20:21:19 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 20:21:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:21:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:21:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:21:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:21:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:21:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:21:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:21:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:21:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:21:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:21:25 [DEBUG   ] Finished updating board flags in 1.450 seconds
2015-11-09 20:21:39 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 20:21:59 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 20:22:19 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 20:22:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:22:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:22:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:22:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:22:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:22:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:22:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:22:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:22:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:22:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:22:25 [DEBUG   ] Finished updating board flags in 1.444 seconds
2015-11-09 20:22:39 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 20:22:48 [DEBUG   ] Got command RPT from MCS: ref #530451
2015-11-09 20:22:48 [DEBUG   ] summary = ERROR
2015-11-09 20:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530451  13 57335 73368623 A  ERRORERROR'
2015-11-09 20:22:48 [DEBUG   ] Got command RPT from MCS: ref #530452
2015-11-09 20:22:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530452  84 57335 73368623 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:22:59 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-09 20:23:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 20:23:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:23:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:23:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:23:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:23:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:23:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:23:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:23:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:23:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:23:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:23:25 [DEBUG   ] Finished updating board flags in 1.452 seconds
2015-11-09 20:23:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 20:23:59 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 20:24:20 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 20:24:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:24:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:24:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:24:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:24:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:24:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:24:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:24:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:24:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:24:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:24:25 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 20:24:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 20:24:25 [DEBUG   ] Finished updating board flags in 1.447 seconds
2015-11-09 20:24:40 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 20:25:00 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 20:25:14 [DEBUG   ] Got command RPT from MCS: ref #530472
2015-11-09 20:25:14 [DEBUG   ] summary = ERROR
2015-11-09 20:25:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530472  13 57335 73514603 A  ERRORERROR'
2015-11-09 20:25:14 [DEBUG   ] Got command RPT from MCS: ref #530473
2015-11-09 20:25:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:25:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530473  84 57335 73514807 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:25:15 [DEBUG   ] Got command RPT from MCS: ref #530474
2015-11-09 20:25:15 [DEBUG   ] summary = ERROR
2015-11-09 20:25:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530474  13 57335 73515011 A  ERRORERROR'
2015-11-09 20:25:15 [DEBUG   ] Got command RPT from MCS: ref #530475
2015-11-09 20:25:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:25:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530475  84 57335 73515215 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:25:15 [DEBUG   ] Got command RPT from MCS: ref #530476
2015-11-09 20:25:15 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 20:25:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530476  12 57335 73515407 A  ERRORB‹ÌÍ'
2015-11-09 20:25:20 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-09 20:25:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:25:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:25:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:25:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:25:24 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:25:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:25:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:25:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:25:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:25:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:25:25 [DEBUG   ] Finished updating board flags in 1.472 seconds
2015-11-09 20:25:40 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 20:26:00 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 20:26:20 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 20:26:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:26:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:26:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:26:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:26:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:26:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:26:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:26:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:26:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:26:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:26:26 [DEBUG   ] Finished updating board flags in 2.156 seconds
2015-11-09 20:26:40 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 20:27:00 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 20:27:20 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 20:27:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:27:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:27:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:27:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:27:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:27:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:27:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:27:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:27:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:27:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:27:26 [DEBUG   ] Finished updating board flags in 2.012 seconds
2015-11-09 20:27:40 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 20:27:48 [DEBUG   ] Got command RPT from MCS: ref #530485
2015-11-09 20:27:48 [DEBUG   ] summary = ERROR
2015-11-09 20:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530485  13 57335 73668627 A  ERRORERROR'
2015-11-09 20:27:48 [DEBUG   ] Got command RPT from MCS: ref #530486
2015-11-09 20:27:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530486  84 57335 73668627 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:28:00 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-09 20:28:20 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 20:28:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:28:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:28:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:28:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:28:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:28:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:28:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:28:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:28:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:28:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:28:26 [DEBUG   ] Finished updating board flags in 1.984 seconds
2015-11-09 20:28:40 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 20:29:00 [DEBUG   ] Finished updating ant stats in 0.808 seconds
2015-11-09 20:29:20 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 20:29:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:29:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:29:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:29:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:29:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:29:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:29:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:29:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:29:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:29:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:29:26 [DEBUG   ] Finished updating board flags in 1.809 seconds
2015-11-09 20:29:40 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 20:30:00 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 20:30:14 [DEBUG   ] Got command RPT from MCS: ref #530505
2015-11-09 20:30:14 [DEBUG   ] summary = ERROR
2015-11-09 20:30:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530505  13 57335 73814695 A  ERRORERROR'
2015-11-09 20:30:14 [DEBUG   ] Got command RPT from MCS: ref #530506
2015-11-09 20:30:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:30:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530506  84 57335 73814899 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:30:15 [DEBUG   ] Got command RPT from MCS: ref #530507
2015-11-09 20:30:15 [DEBUG   ] summary = ERROR
2015-11-09 20:30:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530507  13 57335 73815103 A  ERRORERROR'
2015-11-09 20:30:15 [DEBUG   ] Got command RPT from MCS: ref #530508
2015-11-09 20:30:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:30:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530508  84 57335 73815295 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:30:15 [DEBUG   ] Got command RPT from MCS: ref #530509
2015-11-09 20:30:15 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 20:30:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530509  12 57335 73815499 A  ERRORB‹ÌÍ'
2015-11-09 20:30:20 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2015-11-09 20:30:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:30:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:30:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:30:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:30:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:30:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:30:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:30:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:30:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:30:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:30:26 [DEBUG   ] Finished updating board flags in 1.821 seconds
2015-11-09 20:30:40 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 20:31:01 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 20:31:21 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 20:31:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:31:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:31:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:31:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:31:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:31:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:31:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:31:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:31:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:31:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:31:26 [DEBUG   ] Finished updating board flags in 1.807 seconds
2015-11-09 20:31:41 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 20:32:01 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 20:32:21 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 20:32:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:32:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:32:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:32:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:32:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:32:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:32:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:32:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:32:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:32:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:32:26 [DEBUG   ] Finished updating board flags in 1.776 seconds
2015-11-09 20:32:41 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 20:32:48 [DEBUG   ] Got command RPT from MCS: ref #530513
2015-11-09 20:32:48 [DEBUG   ] summary = ERROR
2015-11-09 20:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530513  13 57335 73968631 A  ERRORERROR'
2015-11-09 20:32:48 [DEBUG   ] Got command RPT from MCS: ref #530514
2015-11-09 20:32:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530514  84 57335 73968631 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:33:01 [DEBUG   ] Finished updating ant stats in 1.008 seconds
2015-11-09 20:33:21 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 20:33:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:33:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:33:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:33:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:33:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:33:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 20:33:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:33:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:33:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:33:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:33:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:33:26 [DEBUG   ] Finished updating board flags in 1.608 seconds
2015-11-09 20:33:41 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 20:34:01 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 20:34:21 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 20:34:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:34:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:34:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:34:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:34:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:34:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:34:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:34:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:34:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:34:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:34:26 [DEBUG   ] Finished updating board flags in 1.604 seconds
2015-11-09 20:34:41 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-09 20:35:01 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 20:35:14 [DEBUG   ] Got command RPT from MCS: ref #530531
2015-11-09 20:35:14 [DEBUG   ] summary = ERROR
2015-11-09 20:35:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530531  13 57335 74114791 A  ERRORERROR'
2015-11-09 20:35:14 [DEBUG   ] Got command RPT from MCS: ref #530532
2015-11-09 20:35:14 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:35:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530532  84 57335 74114983 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:35:15 [DEBUG   ] Got command RPT from MCS: ref #530533
2015-11-09 20:35:15 [DEBUG   ] summary = ERROR
2015-11-09 20:35:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530533  13 57335 74115187 A  ERRORERROR'
2015-11-09 20:35:15 [DEBUG   ] Got command RPT from MCS: ref #530534
2015-11-09 20:35:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:35:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530534  84 57335 74115391 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:35:15 [DEBUG   ] Got command RPT from MCS: ref #530535
2015-11-09 20:35:15 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 20:35:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530535  12 57335 74115595 A  ERRORB�ÌÍ'
2015-11-09 20:35:21 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 20:35:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:35:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:35:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:35:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:35:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:35:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:35:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:35:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:35:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:35:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:35:26 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 20:35:26 [DEBUG   ] Finished updating board flags in 1.586 seconds
2015-11-09 20:35:41 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 20:36:01 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 20:36:21 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 20:36:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:36:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:36:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:36:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:36:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:36:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:36:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:36:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:36:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:36:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:36:26 [DEBUG   ] Finished updating board flags in 1.417 seconds
2015-11-09 20:36:41 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 20:37:01 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 20:37:21 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 20:37:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:37:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:37:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:37:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:37:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:37:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:37:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:37:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:37:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:37:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:37:26 [DEBUG   ] Finished updating board flags in 1.429 seconds
2015-11-09 20:37:41 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 20:37:48 [DEBUG   ] Got command RPT from MCS: ref #530544
2015-11-09 20:37:48 [DEBUG   ] summary = ERROR
2015-11-09 20:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530544  13 57335 74268627 A  ERRORERROR'
2015-11-09 20:37:48 [DEBUG   ] Got command RPT from MCS: ref #530545
2015-11-09 20:37:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530545  84 57335 74268627 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:38:01 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 20:38:21 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 20:38:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:38:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:38:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:38:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:38:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:38:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:38:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:38:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:38:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:38:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:38:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 20:38:26 [DEBUG   ] Finished updating board flags in 1.540 seconds
2015-11-09 20:38:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 20:39:01 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 20:39:21 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 20:39:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:39:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:39:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:39:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:39:25 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:39:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:39:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:39:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:39:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:39:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:39:27 [DEBUG   ] Finished updating board flags in 1.580 seconds
2015-11-09 20:39:42 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 20:40:02 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 20:40:14 [DEBUG   ] Got command RPT from MCS: ref #530564
2015-11-09 20:40:14 [DEBUG   ] summary = ERROR
2015-11-09 20:40:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530564  13 57335 74414863 A  ERRORERROR'
2015-11-09 20:40:15 [DEBUG   ] Got command RPT from MCS: ref #530565
2015-11-09 20:40:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:40:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530565  84 57335 74415067 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:40:15 [DEBUG   ] Got command RPT from MCS: ref #530566
2015-11-09 20:40:15 [DEBUG   ] summary = ERROR
2015-11-09 20:40:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530566  13 57335 74415271 A  ERRORERROR'
2015-11-09 20:40:15 [DEBUG   ] Got command RPT from MCS: ref #530567
2015-11-09 20:40:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:40:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530567  84 57335 74415463 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:40:15 [DEBUG   ] Got command RPT from MCS: ref #530568
2015-11-09 20:40:15 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 20:40:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530568  12 57335 74415667 A  ERRORB�ÌÍ'
2015-11-09 20:40:22 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 20:40:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:40:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:40:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:40:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:40:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:40:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:40:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:40:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:40:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:40:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:40:27 [DEBUG   ] Finished updating board flags in 2.146 seconds
2015-11-09 20:40:42 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 20:41:03 [DEBUG   ] Finished updating ant stats in 1.803 seconds
2015-11-09 20:41:22 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-09 20:41:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:41:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:41:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:41:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:41:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:41:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:41:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:41:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:41:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:41:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:41:27 [DEBUG   ] Finished updating board flags in 1.897 seconds
2015-11-09 20:41:42 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 20:42:02 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 20:42:23 [DEBUG   ] Finished updating ant stats in 1.600 seconds
2015-11-09 20:42:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:42:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:42:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:42:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:42:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:42:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:42:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:42:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:42:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:42:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:42:27 [DEBUG   ] Finished updating board flags in 1.822 seconds
2015-11-09 20:42:43 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 20:42:48 [DEBUG   ] Got command RPT from MCS: ref #530572
2015-11-09 20:42:48 [DEBUG   ] summary = ERROR
2015-11-09 20:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530572  13 57335 74568635 A  ERRORERROR'
2015-11-09 20:42:48 [DEBUG   ] Got command RPT from MCS: ref #530573
2015-11-09 20:42:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530573  84 57335 74568635 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:43:03 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 20:43:23 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 20:43:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:43:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:43:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:43:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:43:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:43:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:43:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:43:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:43:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:43:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:43:27 [DEBUG   ] Finished updating board flags in 1.784 seconds
2015-11-09 20:43:43 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 20:44:03 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 20:44:23 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 20:44:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:44:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:44:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:44:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:44:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:44:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:44:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:44:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:44:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:44:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:44:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 20:44:27 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 20:44:27 [DEBUG   ] Finished updating board flags in 1.613 seconds
2015-11-09 20:44:43 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 20:45:03 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 20:45:14 [DEBUG   ] Got command RPT from MCS: ref #530593
2015-11-09 20:45:14 [DEBUG   ] summary = ERROR
2015-11-09 20:45:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530593  13 57335 74714947 A  ERRORERROR'
2015-11-09 20:45:15 [DEBUG   ] Got command RPT from MCS: ref #530594
2015-11-09 20:45:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:45:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530594  84 57335 74715151 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:45:15 [DEBUG   ] Got command RPT from MCS: ref #530595
2015-11-09 20:45:15 [DEBUG   ] summary = ERROR
2015-11-09 20:45:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530595  13 57335 74715355 A  ERRORERROR'
2015-11-09 20:45:15 [DEBUG   ] Got command RPT from MCS: ref #530596
2015-11-09 20:45:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:45:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530596  84 57335 74715559 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:45:15 [DEBUG   ] Got command RPT from MCS: ref #530597
2015-11-09 20:45:15 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 20:45:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530597  12 57335 74715751 A  ERRORB‹ÌÍ'
2015-11-09 20:45:23 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 20:45:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:45:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:45:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:45:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:45:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:45:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:45:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:45:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:45:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:45:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:45:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 20:45:27 [DEBUG   ] Finished updating board flags in 1.581 seconds
2015-11-09 20:45:43 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 20:46:03 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 20:46:23 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 20:46:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:46:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:46:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:46:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:46:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:46:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:46:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:46:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:46:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:46:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:46:27 [DEBUG   ] Finished updating board flags in 1.416 seconds
2015-11-09 20:46:43 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 20:47:03 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 20:47:23 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 20:47:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:47:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:47:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:47:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:47:26 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:47:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:47:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:47:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:47:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:47:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:47:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 20:47:27 [DEBUG   ] Finished updating board flags in 1.393 seconds
2015-11-09 20:47:43 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 20:47:48 [DEBUG   ] Got command RPT from MCS: ref #530606
2015-11-09 20:47:48 [DEBUG   ] summary = ERROR
2015-11-09 20:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530606  13 57335 74868631 A  ERRORERROR'
2015-11-09 20:47:48 [DEBUG   ] Got command RPT from MCS: ref #530607
2015-11-09 20:47:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530607  84 57335 74868631 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:48:03 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 20:48:23 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 20:48:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:48:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:48:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:48:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:48:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:48:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:48:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:48:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:48:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:48:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:48:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 20:48:28 [DEBUG   ] Finished updating board flags in 2.221 seconds
2015-11-09 20:48:43 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 20:49:03 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 20:49:23 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 20:49:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:49:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:49:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:49:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:49:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:49:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:49:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:49:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:49:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:49:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:49:28 [DEBUG   ] Finished updating board flags in 2.179 seconds
2015-11-09 20:49:43 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 20:50:03 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 20:50:15 [DEBUG   ] Got command RPT from MCS: ref #530623
2015-11-09 20:50:15 [DEBUG   ] summary = ERROR
2015-11-09 20:50:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530623  13 57335 75015035 A  ERRORERROR'
2015-11-09 20:50:15 [DEBUG   ] Got command RPT from MCS: ref #530624
2015-11-09 20:50:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:50:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530624  84 57335 75015239 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:50:15 [DEBUG   ] Got command RPT from MCS: ref #530625
2015-11-09 20:50:15 [DEBUG   ] summary = ERROR
2015-11-09 20:50:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530625  13 57335 75015443 A  ERRORERROR'
2015-11-09 20:50:15 [DEBUG   ] Got command RPT from MCS: ref #530626
2015-11-09 20:50:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:50:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530626  84 57335 75015647 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:50:15 [DEBUG   ] Got command RPT from MCS: ref #530627
2015-11-09 20:50:15 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 20:50:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530627  12 57335 75015839 A  ERRORBŠÌÍ'
2015-11-09 20:50:23 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 20:50:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:50:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:50:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:50:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:50:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:50:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:50:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:50:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:50:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:50:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:50:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 20:50:28 [DEBUG   ] Finished updating board flags in 2.099 seconds
2015-11-09 20:50:44 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2015-11-09 20:51:04 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 20:51:24 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 20:51:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:51:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:51:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:51:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:51:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:51:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:51:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:51:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:51:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:51:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:51:28 [DEBUG   ] Finished updating board flags in 1.901 seconds
2015-11-09 20:51:44 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2015-11-09 20:52:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 20:52:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 20:52:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:52:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:52:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:52:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:52:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:52:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:52:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:52:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:52:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:52:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:52:28 [DEBUG   ] Finished updating board flags in 1.817 seconds
2015-11-09 20:52:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 20:52:48 [DEBUG   ] Got command RPT from MCS: ref #530631
2015-11-09 20:52:48 [DEBUG   ] summary = ERROR
2015-11-09 20:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530631  13 57335 75168639 A  ERRORERROR'
2015-11-09 20:52:48 [DEBUG   ] Got command RPT from MCS: ref #530632
2015-11-09 20:52:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530632  84 57335 75168639 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:53:04 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-09 20:53:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 20:53:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:53:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:53:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:53:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:53:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:53:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:53:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:53:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:53:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:53:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:53:28 [DEBUG   ] Finished updating board flags in 1.792 seconds
2015-11-09 20:53:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 20:54:04 [DEBUG   ] Finished updating ant stats in 0.784 seconds
2015-11-09 20:54:24 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 20:54:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:54:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:54:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:54:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:54:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:54:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:54:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:54:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:54:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:54:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:54:28 [DEBUG   ] Finished updating board flags in 1.608 seconds
2015-11-09 20:54:45 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 20:55:05 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 20:55:15 [DEBUG   ] Got command RPT from MCS: ref #530652
2015-11-09 20:55:15 [DEBUG   ] summary = ERROR
2015-11-09 20:55:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530652  13 57335 75315127 A  ERRORERROR'
2015-11-09 20:55:15 [DEBUG   ] Got command RPT from MCS: ref #530653
2015-11-09 20:55:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:55:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530653  84 57335 75315319 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:55:15 [DEBUG   ] Got command RPT from MCS: ref #530654
2015-11-09 20:55:15 [DEBUG   ] summary = ERROR
2015-11-09 20:55:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530654  13 57335 75315523 A  ERRORERROR'
2015-11-09 20:55:15 [DEBUG   ] Got command RPT from MCS: ref #530655
2015-11-09 20:55:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:55:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530655  84 57335 75315727 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:55:15 [DEBUG   ] Got command RPT from MCS: ref #530656
2015-11-09 20:55:15 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 20:55:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530656  12 57335 75315931 A  ERRORB�ÌÍ'
2015-11-09 20:55:25 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 20:55:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:55:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:55:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:55:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:55:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:55:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:55:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:55:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:55:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:55:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:55:28 [DEBUG   ] Finished updating board flags in 1.607 seconds
2015-11-09 20:55:45 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 20:56:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 20:56:25 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 20:56:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:56:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:56:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:56:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:56:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:56:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:56:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:56:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:56:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:56:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:56:28 [DEBUG   ] Finished updating board flags in 1.591 seconds
2015-11-09 20:56:45 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2015-11-09 20:57:05 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 20:57:25 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 20:57:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:57:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:57:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:57:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:57:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:57:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:57:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:57:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:57:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:57:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:57:28 [DEBUG   ] Finished updating board flags in 1.407 seconds
2015-11-09 20:57:45 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 20:57:48 [DEBUG   ] Got command RPT from MCS: ref #530665
2015-11-09 20:57:48 [DEBUG   ] summary = ERROR
2015-11-09 20:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530665  13 57335 75468631 A  ERRORERROR'
2015-11-09 20:57:48 [DEBUG   ] Got command RPT from MCS: ref #530666
2015-11-09 20:57:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 20:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530666  84 57335 75468631 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 20:58:05 [DEBUG   ] Finished updating ant stats in 0.807 seconds
2015-11-09 20:58:25 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 20:58:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:58:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:58:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:58:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:58:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:58:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:58:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:58:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:58:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:58:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:58:28 [DEBUG   ] Finished updating board flags in 1.387 seconds
2015-11-09 20:58:45 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-09 20:59:05 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 20:59:26 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 20:59:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 20:59:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 20:59:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 20:59:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 20:59:27 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 20:59:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 20:59:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 20:59:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 20:59:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 20:59:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 20:59:28 [DEBUG   ] Finished updating board flags in 1.370 seconds
2015-11-09 20:59:46 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-09 21:00:06 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 21:00:15 [DEBUG   ] Got command RPT from MCS: ref #530685
2015-11-09 21:00:15 [DEBUG   ] summary = ERROR
2015-11-09 21:00:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530685  13 57335 75615203 A  ERRORERROR'
2015-11-09 21:00:15 [DEBUG   ] Got command RPT from MCS: ref #530686
2015-11-09 21:00:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:00:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530686  84 57335 75615407 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:00:15 [DEBUG   ] Got command RPT from MCS: ref #530687
2015-11-09 21:00:15 [DEBUG   ] summary = ERROR
2015-11-09 21:00:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530687  13 57335 75615611 A  ERRORERROR'
2015-11-09 21:00:15 [DEBUG   ] Got command RPT from MCS: ref #530688
2015-11-09 21:00:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:00:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530688  84 57335 75615803 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:00:16 [DEBUG   ] Got command RPT from MCS: ref #530689
2015-11-09 21:00:16 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 21:00:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530689  12 57335 75616007 A  ERRORBŒÌÍ'
2015-11-09 21:00:26 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 21:00:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:00:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:00:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:00:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:00:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:00:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:00:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:00:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:00:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:00:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:00:29 [DEBUG   ] Finished updating board flags in 2.053 seconds
2015-11-09 21:00:46 [DEBUG   ] Finished updating ant stats in 1.211 seconds
2015-11-09 21:01:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 21:01:26 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 21:01:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:01:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:01:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:01:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:01:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:01:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:01:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:01:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:01:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:01:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:01:29 [DEBUG   ] Finished updating board flags in 1.985 seconds
2015-11-09 21:01:46 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 21:02:06 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 21:02:26 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:02:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:02:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:02:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:02:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:02:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:02:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:02:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:02:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:02:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:02:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:02:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:02:29 [DEBUG   ] Finished updating board flags in 1.813 seconds
2015-11-09 21:02:46 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 21:02:48 [DEBUG   ] Got command RPT from MCS: ref #530693
2015-11-09 21:02:48 [DEBUG   ] summary = ERROR
2015-11-09 21:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530693  13 57335 75768635 A  ERRORERROR'
2015-11-09 21:02:48 [DEBUG   ] Got command RPT from MCS: ref #530694
2015-11-09 21:02:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530694  84 57335 75768635 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:03:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 21:03:26 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 21:03:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:03:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:03:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:03:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:03:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:03:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:03:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:03:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:03:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:03:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:03:29 [DEBUG   ] Finished updating board flags in 1.797 seconds
2015-11-09 21:03:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 21:04:06 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2015-11-09 21:04:27 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-09 21:04:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:04:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:04:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:04:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:04:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:04:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 21:04:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:04:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:04:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:04:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:04:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:04:29 [DEBUG   ] Finished updating board flags in 1.603 seconds
2015-11-09 21:04:47 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 21:05:07 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-09 21:05:15 [DEBUG   ] Got command RPT from MCS: ref #530711
2015-11-09 21:05:15 [DEBUG   ] summary = ERROR
2015-11-09 21:05:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530711  13 57335 75915295 A  ERRORERROR'
2015-11-09 21:05:15 [DEBUG   ] Got command RPT from MCS: ref #530712
2015-11-09 21:05:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:05:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530712  84 57335 75915487 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:05:15 [DEBUG   ] Got command RPT from MCS: ref #530713
2015-11-09 21:05:15 [DEBUG   ] summary = ERROR
2015-11-09 21:05:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530713  13 57335 75915691 A  ERRORERROR'
2015-11-09 21:05:15 [DEBUG   ] Got command RPT from MCS: ref #530714
2015-11-09 21:05:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:05:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530714  84 57335 75915895 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:05:16 [DEBUG   ] Got command RPT from MCS: ref #530715
2015-11-09 21:05:16 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 21:05:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530715  12 57335 75916099 A  ERRORB‹ÌÍ'
2015-11-09 21:05:27 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 21:05:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:05:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:05:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:05:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:05:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:05:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:05:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:05:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:05:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:05:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:05:29 [DEBUG   ] Finished updating board flags in 1.611 seconds
2015-11-09 21:05:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 21:06:07 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 21:06:27 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 21:06:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:06:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:06:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:06:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:06:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:06:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:06:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:06:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:06:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:06:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:06:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:06:29 [DEBUG   ] Finished updating board flags in 1.668 seconds
2015-11-09 21:06:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 21:07:07 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 21:07:27 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 21:07:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:07:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:07:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:07:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:07:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:07:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:07:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:07:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:07:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:07:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:07:29 [DEBUG   ] Finished updating board flags in 1.625 seconds
2015-11-09 21:07:47 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-09 21:07:48 [DEBUG   ] Got command RPT from MCS: ref #530724
2015-11-09 21:07:48 [DEBUG   ] summary = ERROR
2015-11-09 21:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530724  13 57335 76068643 A  ERRORERROR'
2015-11-09 21:07:48 [DEBUG   ] Got command RPT from MCS: ref #530725
2015-11-09 21:07:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530725  84 57335 76068643 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:08:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 21:08:27 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 21:08:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:08:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:08:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:08:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:08:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:08:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:08:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:08:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:08:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:08:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:08:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 21:08:29 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 21:08:29 [DEBUG   ] Finished updating board flags in 1.507 seconds
2015-11-09 21:08:47 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 21:09:07 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 21:09:27 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 21:09:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:09:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:09:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:09:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:09:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:09:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:09:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:09:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:09:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:09:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:09:29 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 21:09:29 [DEBUG   ] Finished updating board flags in 1.483 seconds
2015-11-09 21:09:47 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 21:10:07 [DEBUG   ] Finished updating ant stats in 0.924 seconds
2015-11-09 21:10:15 [DEBUG   ] Got command RPT from MCS: ref #530744
2015-11-09 21:10:15 [DEBUG   ] summary = ERROR
2015-11-09 21:10:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530744  13 57335 76215379 A  ERRORERROR'
2015-11-09 21:10:15 [DEBUG   ] Got command RPT from MCS: ref #530745
2015-11-09 21:10:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:10:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530745  84 57335 76215583 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:10:15 [DEBUG   ] Got command RPT from MCS: ref #530746
2015-11-09 21:10:15 [DEBUG   ] summary = ERROR
2015-11-09 21:10:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530746  13 57335 76215775 A  ERRORERROR'
2015-11-09 21:10:15 [DEBUG   ] Got command RPT from MCS: ref #530747
2015-11-09 21:10:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:10:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530747  84 57335 76215979 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:10:16 [DEBUG   ] Got command RPT from MCS: ref #530748
2015-11-09 21:10:16 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 21:10:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530748  12 57335 76216175 A  ERRORB‹ÌÍ'
2015-11-09 21:10:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:10:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:10:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:10:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:10:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:10:28 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:10:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:10:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:10:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:10:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:10:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:10:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:10:29 [DEBUG   ] Finished updating board flags in 1.484 seconds
2015-11-09 21:10:47 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:11:07 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:11:27 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 21:11:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:11:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:11:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:11:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:11:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:11:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:11:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:11:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:11:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:11:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:11:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 21:11:30 [DEBUG   ] Finished updating board flags in 2.221 seconds
2015-11-09 21:11:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:12:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:12:28 [DEBUG   ] Finished updating ant stats in 1.800 seconds
2015-11-09 21:12:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:12:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:12:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:12:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:12:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:12:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:12:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:12:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:12:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:12:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:12:30 [DEBUG   ] Finished updating board flags in 2.168 seconds
2015-11-09 21:12:47 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 21:12:48 [DEBUG   ] Got command RPT from MCS: ref #530752
2015-11-09 21:12:48 [DEBUG   ] summary = ERROR
2015-11-09 21:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530752  13 57335 76368635 A  ERRORERROR'
2015-11-09 21:12:48 [DEBUG   ] Got command RPT from MCS: ref #530753
2015-11-09 21:12:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530753  84 57335 76368635 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:13:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:13:27 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:13:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:13:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:13:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:13:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:13:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:13:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:13:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:13:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:13:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:13:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:13:30 [DEBUG   ] Finished updating board flags in 2.112 seconds
2015-11-09 21:13:47 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:14:07 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 21:14:27 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 21:14:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:14:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:14:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:14:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:14:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:14:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:14:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:14:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:14:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:14:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:14:30 [DEBUG   ] Finished updating board flags in 1.884 seconds
2015-11-09 21:14:47 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:15:07 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:15:15 [DEBUG   ] Got command RPT from MCS: ref #530773
2015-11-09 21:15:15 [DEBUG   ] summary = ERROR
2015-11-09 21:15:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530773  13 57335 76515463 A  ERRORERROR'
2015-11-09 21:15:15 [DEBUG   ] Got command RPT from MCS: ref #530774
2015-11-09 21:15:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:15:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530774  84 57335 76515667 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:15:15 [DEBUG   ] Got command RPT from MCS: ref #530775
2015-11-09 21:15:15 [DEBUG   ] summary = ERROR
2015-11-09 21:15:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530775  13 57335 76515871 A  ERRORERROR'
2015-11-09 21:15:16 [DEBUG   ] Got command RPT from MCS: ref #530776
2015-11-09 21:15:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:15:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530776  84 57335 76516063 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:15:16 [DEBUG   ] Got command RPT from MCS: ref #530777
2015-11-09 21:15:16 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 21:15:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530777  12 57335 76516267 A  ERRORB�ÌÍ'
2015-11-09 21:15:27 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-09 21:15:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:15:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:15:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:15:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:15:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:15:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:15:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:15:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:15:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:15:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:15:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa34, chip 1
2015-11-09 21:15:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 21:15:30 [DEBUG   ] Finished updating board flags in 1.825 seconds
2015-11-09 21:15:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 21:16:07 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 21:16:27 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 21:16:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:16:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:16:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:16:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:16:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:16:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:16:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:16:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:16:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:16:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:16:30 [DEBUG   ] Finished updating board flags in 1.771 seconds
2015-11-09 21:16:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 21:17:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 21:17:27 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 21:17:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:17:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:17:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:17:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:17:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:17:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:17:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:17:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:17:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:17:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:17:30 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:17:30 [DEBUG   ] Finished updating board flags in 1.623 seconds
2015-11-09 21:17:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 21:17:48 [DEBUG   ] Got command RPT from MCS: ref #530786
2015-11-09 21:17:48 [DEBUG   ] summary = ERROR
2015-11-09 21:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530786  13 57335 76668647 A  ERRORERROR'
2015-11-09 21:17:48 [DEBUG   ] Got command RPT from MCS: ref #530787
2015-11-09 21:17:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530787  84 57335 76668647 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:18:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 21:18:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 21:18:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:18:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:18:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:18:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:18:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:18:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:18:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:18:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:18:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:18:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:18:30 [DEBUG   ] Finished updating board flags in 1.617 seconds
2015-11-09 21:18:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 21:19:07 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 21:19:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 21:19:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:19:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:19:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:19:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:19:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:19:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:19:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:19:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:19:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:19:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:19:30 [DEBUG   ] Finished updating board flags in 1.631 seconds
2015-11-09 21:19:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 21:20:07 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 21:20:15 [DEBUG   ] Got command RPT from MCS: ref #530803
2015-11-09 21:20:15 [DEBUG   ] summary = ERROR
2015-11-09 21:20:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530803  13 57335 76815543 A  ERRORERROR'
2015-11-09 21:20:15 [DEBUG   ] Got command RPT from MCS: ref #530804
2015-11-09 21:20:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:20:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530804  84 57335 76815747 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:20:15 [DEBUG   ] Got command RPT from MCS: ref #530805
2015-11-09 21:20:15 [DEBUG   ] summary = ERROR
2015-11-09 21:20:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530805  13 57335 76815939 A  ERRORERROR'
2015-11-09 21:20:16 [DEBUG   ] Got command RPT from MCS: ref #530806
2015-11-09 21:20:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:20:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530806  84 57335 76816143 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:20:16 [DEBUG   ] Got command RPT from MCS: ref #530807
2015-11-09 21:20:16 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 21:20:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530807  12 57335 76816347 A  ERRORBŒÌÍ'
2015-11-09 21:20:27 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 21:20:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:20:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:20:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:20:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:20:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:20:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:20:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:20:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:20:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:20:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:20:30 [DEBUG   ] Finished updating board flags in 1.642 seconds
2015-11-09 21:20:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 21:21:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 21:21:27 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 21:21:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:21:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:21:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:21:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:21:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:21:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:21:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:21:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:21:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:21:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:21:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:21:30 [DEBUG   ] Finished updating board flags in 1.507 seconds
2015-11-09 21:21:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 21:22:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 21:22:28 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-09 21:22:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:22:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:22:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:22:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:22:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:22:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:22:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:22:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:22:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:22:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:22:30 [DEBUG   ] Finished updating board flags in 1.511 seconds
2015-11-09 21:22:48 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-09 21:22:48 [DEBUG   ] Got command RPT from MCS: ref #530811
2015-11-09 21:22:48 [DEBUG   ] summary = ERROR
2015-11-09 21:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530811  13 57335 76968643 A  ERRORERROR'
2015-11-09 21:22:48 [DEBUG   ] Got command RPT from MCS: ref #530812
2015-11-09 21:22:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530812  84 57335 76968643 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:23:08 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 21:23:28 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 21:23:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:23:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:23:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:23:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:23:29 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:23:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:23:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:23:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:23:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:23:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:23:30 [DEBUG   ] Finished updating board flags in 1.513 seconds
2015-11-09 21:23:48 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 21:24:08 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 21:24:28 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 21:24:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:24:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:24:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:24:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:24:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:24:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:24:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:24:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:24:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:24:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:24:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:24:31 [DEBUG   ] Finished updating board flags in 2.207 seconds
2015-11-09 21:24:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 21:25:08 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 21:25:15 [DEBUG   ] Got command RPT from MCS: ref #530832
2015-11-09 21:25:15 [DEBUG   ] summary = ERROR
2015-11-09 21:25:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530832  13 57335 77115623 A  ERRORERROR'
2015-11-09 21:25:15 [DEBUG   ] Got command RPT from MCS: ref #530833
2015-11-09 21:25:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:25:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530833  84 57335 77115827 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:25:16 [DEBUG   ] Got command RPT from MCS: ref #530834
2015-11-09 21:25:16 [DEBUG   ] summary = ERROR
2015-11-09 21:25:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530834  13 57335 77116031 A  ERRORERROR'
2015-11-09 21:25:16 [DEBUG   ] Got command RPT from MCS: ref #530835
2015-11-09 21:25:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:25:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530835  84 57335 77116223 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:25:16 [DEBUG   ] Got command RPT from MCS: ref #530836
2015-11-09 21:25:16 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 21:25:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530836  12 57335 77116427 A  ERRORB‹ÌÍ'
2015-11-09 21:25:28 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 21:25:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:25:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:25:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:25:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:25:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:25:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:25:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:25:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:25:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:25:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:25:31 [DEBUG   ] Finished updating board flags in 2.185 seconds
2015-11-09 21:25:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 21:26:08 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 21:26:28 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 21:26:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:26:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:26:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:26:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:26:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:26:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:26:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:26:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:26:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:26:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:26:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:26:31 [DEBUG   ] Finished updating board flags in 2.025 seconds
2015-11-09 21:26:48 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 21:27:08 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-09 21:27:28 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 21:27:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:27:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:27:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:27:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:27:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:27:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:27:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:27:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:27:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:27:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:27:31 [DEBUG   ] Finished updating board flags in 1.983 seconds
2015-11-09 21:27:48 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 21:27:48 [DEBUG   ] Got command RPT from MCS: ref #530845
2015-11-09 21:27:48 [DEBUG   ] summary = ERROR
2015-11-09 21:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530845  13 57335 77268647 A  ERRORERROR'
2015-11-09 21:27:48 [DEBUG   ] Got command RPT from MCS: ref #530846
2015-11-09 21:27:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530846  84 57335 77268647 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:28:08 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 21:28:28 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 21:28:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:28:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:28:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:28:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:28:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:28:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:28:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:28:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:28:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:28:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:28:31 [DEBUG   ] Finished updating board flags in 1.901 seconds
2015-11-09 21:28:48 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 21:29:08 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 21:29:28 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 21:29:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:29:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:29:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:29:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:29:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:29:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:29:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:29:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:29:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:29:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:29:31 [DEBUG   ] Finished updating board flags in 1.683 seconds
2015-11-09 21:29:48 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 21:30:08 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 21:30:15 [DEBUG   ] Got command RPT from MCS: ref #530865
2015-11-09 21:30:15 [DEBUG   ] summary = ERROR
2015-11-09 21:30:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530865  13 57335 77415711 A  ERRORERROR'
2015-11-09 21:30:15 [DEBUG   ] Got command RPT from MCS: ref #530866
2015-11-09 21:30:15 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:30:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530866  84 57335 77415915 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:30:16 [DEBUG   ] Got command RPT from MCS: ref #530867
2015-11-09 21:30:16 [DEBUG   ] summary = ERROR
2015-11-09 21:30:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530867  13 57335 77416119 A  ERRORERROR'
2015-11-09 21:30:16 [DEBUG   ] Got command RPT from MCS: ref #530868
2015-11-09 21:30:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:30:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530868  84 57335 77416323 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:30:16 [DEBUG   ] Got command RPT from MCS: ref #530869
2015-11-09 21:30:16 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 21:30:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530869  12 57335 77416515 A  ERRORBŒÌÍ'
2015-11-09 21:30:28 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 21:30:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:30:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:30:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:30:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:30:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:30:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 21:30:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:30:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:30:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:30:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:30:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:30:31 [DEBUG   ] Finished updating board flags in 1.626 seconds
2015-11-09 21:30:48 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 21:31:08 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 21:31:28 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 21:31:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:31:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:31:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:31:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:31:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:31:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:31:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:31:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:31:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:31:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:31:31 [DEBUG   ] Finished updating board flags in 1.615 seconds
2015-11-09 21:31:48 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 21:32:08 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 21:32:28 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 21:32:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:32:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:32:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:32:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:32:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:32:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:32:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:32:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:32:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:32:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:32:31 [DEBUG   ] Finished updating board flags in 1.576 seconds
2015-11-09 21:32:48 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 21:32:48 [DEBUG   ] Got command RPT from MCS: ref #530873
2015-11-09 21:32:48 [DEBUG   ] summary = ERROR
2015-11-09 21:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530873  13 57335 77568639 A  ERRORERROR'
2015-11-09 21:32:48 [DEBUG   ] Got command RPT from MCS: ref #530874
2015-11-09 21:32:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530874  84 57335 77568639 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:33:08 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 21:33:28 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 21:33:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:33:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:33:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:33:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:33:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:33:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:33:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:33:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:33:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:33:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:33:31 [DEBUG   ] Finished updating board flags in 1.581 seconds
2015-11-09 21:33:48 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 21:34:08 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 21:34:28 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 21:34:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:34:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:34:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:34:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:34:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:34:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:34:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:34:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:34:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:34:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:34:32 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:34:32 [DEBUG   ] Finished updating board flags in 1.601 seconds
2015-11-09 21:34:48 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 21:35:08 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 21:35:15 [DEBUG   ] Got command RPT from MCS: ref #530891
2015-11-09 21:35:15 [DEBUG   ] summary = ERROR
2015-11-09 21:35:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530891  13 57335 77715799 A  ERRORERROR'
2015-11-09 21:35:16 [DEBUG   ] Got command RPT from MCS: ref #530892
2015-11-09 21:35:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:35:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530892  84 57335 77716003 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:35:16 [DEBUG   ] Got command RPT from MCS: ref #530893
2015-11-09 21:35:16 [DEBUG   ] summary = ERROR
2015-11-09 21:35:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530893  13 57335 77716207 A  ERRORERROR'
2015-11-09 21:35:16 [DEBUG   ] Got command RPT from MCS: ref #530894
2015-11-09 21:35:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:35:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530894  84 57335 77716399 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:35:16 [DEBUG   ] Got command RPT from MCS: ref #530895
2015-11-09 21:35:16 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 21:35:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530895  12 57335 77716603 A  ERRORBŒÌÍ'
2015-11-09 21:35:28 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 21:35:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:35:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:35:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:35:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:35:30 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:35:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:35:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:35:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:35:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:35:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:35:31 [DEBUG   ] Finished updating board flags in 1.343 seconds
2015-11-09 21:35:48 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 21:36:08 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-09 21:36:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:36:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:36:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:36:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:36:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:36:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:36:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:36:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:36:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:36:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:36:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:36:32 [DEBUG   ] Finished updating board flags in 2.095 seconds
2015-11-09 21:36:48 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:37:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 21:37:28 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:37:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:37:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:37:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:37:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:37:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:37:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:37:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:37:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:37:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:37:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:37:32 [DEBUG   ] Finished updating board flags in 1.994 seconds
2015-11-09 21:37:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:37:48 [DEBUG   ] Got command RPT from MCS: ref #530904
2015-11-09 21:37:48 [DEBUG   ] summary = ERROR
2015-11-09 21:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530904  13 57335 77868643 A  ERRORERROR'
2015-11-09 21:37:48 [DEBUG   ] Got command RPT from MCS: ref #530905
2015-11-09 21:37:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530905  84 57335 77868643 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:38:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:38:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 21:38:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:38:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:38:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:38:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:38:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:38:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:38:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:38:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:38:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:38:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:38:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 21:38:32 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:38:32 [DEBUG   ] Finished updating board flags in 1.814 seconds
2015-11-09 21:38:48 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:39:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:39:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:39:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:39:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:39:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:39:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:39:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:39:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:39:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:39:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:39:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:39:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:39:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 21:39:32 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 21:39:32 [DEBUG   ] Finished updating board flags in 1.814 seconds
2015-11-09 21:39:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 21:40:08 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:40:15 [DEBUG   ] Got command RPT from MCS: ref #530924
2015-11-09 21:40:15 [DEBUG   ] summary = ERROR
2015-11-09 21:40:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530924  13 57335 78015891 A  ERRORERROR'
2015-11-09 21:40:16 [DEBUG   ] Got command RPT from MCS: ref #530925
2015-11-09 21:40:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:40:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530925  84 57335 78016095 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:40:16 [DEBUG   ] Got command RPT from MCS: ref #530926
2015-11-09 21:40:16 [DEBUG   ] summary = ERROR
2015-11-09 21:40:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530926  13 57335 78016291 A  ERRORERROR'
2015-11-09 21:40:16 [DEBUG   ] Got command RPT from MCS: ref #530927
2015-11-09 21:40:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:40:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530927  84 57335 78016495 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:40:16 [DEBUG   ] Got command RPT from MCS: ref #530928
2015-11-09 21:40:16 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 21:40:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530928  12 57335 78016687 A  ERRORB‹ÌÍ'
2015-11-09 21:40:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:40:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:40:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:40:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:40:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:40:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:40:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:40:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:40:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:40:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:40:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:40:32 [DEBUG   ] Finished updating board flags in 1.880 seconds
2015-11-09 21:40:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 21:41:08 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 21:41:28 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:41:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:41:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:41:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:41:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:41:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:41:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:41:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:41:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:41:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:41:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:41:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:41:32 [DEBUG   ] Finished updating board flags in 1.825 seconds
2015-11-09 21:41:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:42:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:42:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 21:42:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:42:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:42:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:42:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:42:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:42:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:42:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:42:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:42:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:42:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:42:32 [DEBUG   ] Finished updating board flags in 1.671 seconds
2015-11-09 21:42:48 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:42:48 [DEBUG   ] Got command RPT from MCS: ref #530932
2015-11-09 21:42:48 [DEBUG   ] summary = ERROR
2015-11-09 21:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530932  13 57335 78168643 A  ERRORERROR'
2015-11-09 21:42:48 [DEBUG   ] Got command RPT from MCS: ref #530933
2015-11-09 21:42:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530933  84 57335 78168643 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:43:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:43:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:43:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:43:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:43:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:43:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:43:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:43:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:43:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:43:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:43:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:43:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:43:32 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:43:32 [DEBUG   ] Finished updating board flags in 1.620 seconds
2015-11-09 21:43:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 21:44:08 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:44:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:44:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:44:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:44:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:44:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:44:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:44:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:44:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:44:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:44:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:44:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:44:32 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:44:32 [DEBUG   ] Finished updating board flags in 1.585 seconds
2015-11-09 21:44:48 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:45:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 21:45:15 [DEBUG   ] Got command RPT from MCS: ref #530953
2015-11-09 21:45:15 [DEBUG   ] summary = ERROR
2015-11-09 21:45:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530953  13 57335 78315979 A  ERRORERROR'
2015-11-09 21:45:16 [DEBUG   ] Got command RPT from MCS: ref #530954
2015-11-09 21:45:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:45:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530954  84 57335 78316183 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:45:16 [DEBUG   ] Got command RPT from MCS: ref #530955
2015-11-09 21:45:16 [DEBUG   ] summary = ERROR
2015-11-09 21:45:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530955  13 57335 78316375 A  ERRORERROR'
2015-11-09 21:45:16 [DEBUG   ] Got command RPT from MCS: ref #530956
2015-11-09 21:45:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:45:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530956  84 57335 78316579 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:45:16 [DEBUG   ] Got command RPT from MCS: ref #530957
2015-11-09 21:45:16 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 21:45:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530957  12 57335 78316783 A  ERRORBŒÌÍ'
2015-11-09 21:45:28 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:45:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:45:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:45:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:45:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:45:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:45:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:45:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:45:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:45:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:45:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:45:32 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:45:32 [DEBUG   ] Finished updating board flags in 1.454 seconds
2015-11-09 21:45:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:46:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:46:28 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-09 21:46:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:46:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:46:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:46:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:46:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:46:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:46:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:46:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:46:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:46:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:46:32 [DEBUG   ] Finished updating board flags in 1.523 seconds
2015-11-09 21:46:48 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2015-11-09 21:47:09 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 21:47:29 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 21:47:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:47:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:47:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:47:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:47:31 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:47:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 21:47:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:47:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:47:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:47:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:47:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:47:32 [DEBUG   ] Finished updating board flags in 1.538 seconds
2015-11-09 21:47:48 [DEBUG   ] Got command RPT from MCS: ref #530966
2015-11-09 21:47:48 [DEBUG   ] summary = ERROR
2015-11-09 21:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530966  13 57335 78468651 A  ERRORERROR'
2015-11-09 21:47:48 [DEBUG   ] Got command RPT from MCS: ref #530967
2015-11-09 21:47:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530967  84 57335 78468652 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:47:49 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 21:48:09 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-09 21:48:29 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 21:48:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:48:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:48:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:48:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:48:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:48:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:48:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:48:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:48:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:48:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:48:33 [DEBUG   ] Finished updating board flags in 2.160 seconds
2015-11-09 21:48:49 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 21:49:09 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 21:49:29 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-09 21:49:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:49:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:49:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:49:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:49:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:49:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:49:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:49:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:49:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:49:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:49:33 [DEBUG   ] Finished updating board flags in 1.953 seconds
2015-11-09 21:49:49 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 21:50:09 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 21:50:16 [DEBUG   ] Got command RPT from MCS: ref #530983
2015-11-09 21:50:16 [DEBUG   ] summary = ERROR
2015-11-09 21:50:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530983  13 57335 78616071 A  ERRORERROR'
2015-11-09 21:50:16 [DEBUG   ] Got command RPT from MCS: ref #530984
2015-11-09 21:50:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:50:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530984  84 57335 78616263 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:50:16 [DEBUG   ] Got command RPT from MCS: ref #530985
2015-11-09 21:50:16 [DEBUG   ] summary = ERROR
2015-11-09 21:50:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530985  13 57335 78616467 A  ERRORERROR'
2015-11-09 21:50:16 [DEBUG   ] Got command RPT from MCS: ref #530986
2015-11-09 21:50:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:50:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530986  84 57335 78616671 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:50:16 [DEBUG   ] Got command RPT from MCS: ref #530987
2015-11-09 21:50:16 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 21:50:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530987  12 57335 78616875 A  ERRORBŒÌÍ'
2015-11-09 21:50:29 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:50:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:50:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:50:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:50:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:50:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:50:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:50:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:50:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:50:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:50:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:50:33 [DEBUG   ] Finished updating board flags in 1.796 seconds
2015-11-09 21:50:49 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:51:09 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:51:29 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 21:51:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:51:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:51:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:51:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:51:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:51:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:51:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:51:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:51:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:51:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:51:33 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:51:33 [DEBUG   ] Finished updating board flags in 1.627 seconds
2015-11-09 21:51:49 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:52:09 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 21:52:29 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 21:52:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:52:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:52:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:52:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:52:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:52:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:52:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:52:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:52:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:52:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:52:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 21:52:33 [DEBUG   ] Finished updating board flags in 1.621 seconds
2015-11-09 21:52:48 [DEBUG   ] Got command RPT from MCS: ref #530991
2015-11-09 21:52:48 [DEBUG   ] summary = ERROR
2015-11-09 21:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530991  13 57335 78768651 A  ERRORERROR'
2015-11-09 21:52:48 [DEBUG   ] Got command RPT from MCS: ref #530992
2015-11-09 21:52:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   530992  84 57335 78768651 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:52:49 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 21:53:09 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 21:53:29 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 21:53:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:53:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:53:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:53:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:53:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:53:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:53:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:53:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:53:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:53:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:53:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 21:53:33 [DEBUG   ] Finished updating board flags in 1.543 seconds
2015-11-09 21:53:49 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2015-11-09 21:54:10 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-09 21:54:30 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 21:54:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:54:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:54:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:54:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:54:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:54:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:54:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:54:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:54:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:54:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:54:33 [DEBUG   ] Finished updating board flags in 1.416 seconds
2015-11-09 21:54:50 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 21:55:10 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 21:55:16 [DEBUG   ] Got command RPT from MCS: ref #531012
2015-11-09 21:55:16 [DEBUG   ] summary = ERROR
2015-11-09 21:55:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531012  13 57335 78916151 A  ERRORERROR'
2015-11-09 21:55:16 [DEBUG   ] Got command RPT from MCS: ref #531013
2015-11-09 21:55:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:55:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531013  84 57335 78916355 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:55:16 [DEBUG   ] Got command RPT from MCS: ref #531014
2015-11-09 21:55:16 [DEBUG   ] summary = ERROR
2015-11-09 21:55:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531014  13 57335 78916559 A  ERRORERROR'
2015-11-09 21:55:16 [DEBUG   ] Got command RPT from MCS: ref #531015
2015-11-09 21:55:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:55:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531015  84 57335 78916751 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:55:16 [DEBUG   ] Got command RPT from MCS: ref #531016
2015-11-09 21:55:16 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 21:55:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531016  12 57335 78916955 A  ERRORB‹ÌÍ'
2015-11-09 21:55:30 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 21:55:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:55:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:55:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:55:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:55:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:55:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:55:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:55:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:55:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:55:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:55:33 [DEBUG   ] Finished updating board flags in 1.427 seconds
2015-11-09 21:55:50 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-09 21:56:10 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 21:56:30 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 21:56:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:56:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:56:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:56:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:56:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:56:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 21:56:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:56:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:56:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:56:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:56:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:56:33 [DEBUG   ] Finished updating board flags in 1.423 seconds
2015-11-09 21:56:50 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 21:57:10 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-09 21:57:30 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:57:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:57:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:57:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:57:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:57:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:57:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 21:57:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:57:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:57:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:57:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:57:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:57:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 21:57:33 [DEBUG   ] Finished updating board flags in 1.423 seconds
2015-11-09 21:57:48 [DEBUG   ] Got command RPT from MCS: ref #531025
2015-11-09 21:57:48 [DEBUG   ] summary = ERROR
2015-11-09 21:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531025  13 57335 79068651 A  ERRORERROR'
2015-11-09 21:57:48 [DEBUG   ] Got command RPT from MCS: ref #531026
2015-11-09 21:57:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 21:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531026  84 57335 79068651 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 21:57:50 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 21:58:10 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:58:30 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:58:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:58:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:58:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:58:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:58:32 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:58:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:58:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:58:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:58:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:58:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:58:33 [DEBUG   ] Finished updating board flags in 1.536 seconds
2015-11-09 21:58:50 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:59:10 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 21:59:30 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 21:59:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 21:59:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 21:59:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 21:59:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 21:59:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 21:59:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 21:59:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 21:59:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 21:59:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 21:59:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 21:59:34 [DEBUG   ] Finished updating board flags in 2.289 seconds
2015-11-09 21:59:50 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:00:10 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:00:16 [DEBUG   ] Got command RPT from MCS: ref #531045
2015-11-09 22:00:16 [DEBUG   ] summary = ERROR
2015-11-09 22:00:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531045  13 57335 79216243 A  ERRORERROR'
2015-11-09 22:00:16 [DEBUG   ] Got command RPT from MCS: ref #531046
2015-11-09 22:00:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:00:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531046  84 57335 79216447 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:00:16 [DEBUG   ] Got command RPT from MCS: ref #531047
2015-11-09 22:00:16 [DEBUG   ] summary = ERROR
2015-11-09 22:00:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531047  13 57335 79216639 A  ERRORERROR'
2015-11-09 22:00:16 [DEBUG   ] Got command RPT from MCS: ref #531048
2015-11-09 22:00:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:00:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531048  84 57335 79216843 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:00:17 [DEBUG   ] Got command RPT from MCS: ref #531049
2015-11-09 22:00:17 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 22:00:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531049  12 57335 79217047 A  ERRORBŠÌÍ'
2015-11-09 22:00:30 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 22:00:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 22:00:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 22:00:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 22:00:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 22:00:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 22:00:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 22:00:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 22:00:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 22:00:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 22:00:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 22:00:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 22:00:34 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 22:00:34 [DEBUG   ] Finished updating board flags in 2.117 seconds
2015-11-09 22:00:50 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 22:01:10 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:01:30 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 22:01:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 22:01:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 22:01:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 22:01:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 22:01:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 22:01:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 22:01:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 22:01:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 22:01:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 22:01:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 22:01:34 [DEBUG   ] Finished updating board flags in 2.042 seconds
2015-11-09 22:01:50 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 22:02:10 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2015-11-09 22:02:31 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 22:02:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 22:02:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 22:02:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 22:02:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 22:02:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 22:02:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 22:02:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 22:02:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 22:02:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 22:02:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 22:02:34 [DEBUG   ] Finished updating board flags in 1.952 seconds
2015-11-09 22:02:48 [DEBUG   ] Got command RPT from MCS: ref #531053
2015-11-09 22:02:48 [DEBUG   ] summary = ERROR
2015-11-09 22:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531053  13 57335 79368647 A  ERRORERROR'
2015-11-09 22:02:48 [DEBUG   ] Got command RPT from MCS: ref #531054
2015-11-09 22:02:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531054  84 57335 79368647 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:02:51 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 22:03:11 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:03:31 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:03:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998627
2015-11-09 22:03:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998627
2015-11-09 22:03:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998627
2015-11-09 22:03:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998627
2015-11-09 22:03:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998627
2015-11-09 22:03:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998627
2015-11-09 22:03:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998627
2015-11-09 22:03:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998627
2015-11-09 22:03:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998627
2015-11-09 22:03:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998627
2015-11-09 22:03:34 [DEBUG   ] Finished updating board flags in 1.811 seconds
2015-11-09 22:03:51 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:04:11 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 22:04:31 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-09 22:04:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-09 22:04:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-09 22:04:34 [DEBUG   ] Finished updating board flags in 1.804 seconds
2015-11-09 22:04:51 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 22:05:11 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 22:05:16 [DEBUG   ] Got command RPT from MCS: ref #531071
2015-11-09 22:05:16 [DEBUG   ] summary = ERROR
2015-11-09 22:05:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531071  13 57335 79516327 A  ERRORERROR'
2015-11-09 22:05:16 [DEBUG   ] Got command RPT from MCS: ref #531072
2015-11-09 22:05:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:05:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531072  84 57335 79516531 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:05:16 [DEBUG   ] Got command RPT from MCS: ref #531073
2015-11-09 22:05:16 [DEBUG   ] summary = ERROR
2015-11-09 22:05:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531073  13 57335 79516735 A  ERRORERROR'
2015-11-09 22:05:16 [DEBUG   ] Got command RPT from MCS: ref #531074
2015-11-09 22:05:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:05:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531074  84 57335 79516927 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:05:17 [DEBUG   ] Got command RPT from MCS: ref #531075
2015-11-09 22:05:17 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 22:05:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531075  12 57335 79517131 A  ERRORB�ÌÍ'
2015-11-09 22:05:31 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 22:05:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 22:05:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 22:05:34 [DEBUG   ] Finished updating board flags in 1.818 seconds
2015-11-09 22:05:51 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-09 22:06:11 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 22:06:31 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 22:06:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:06:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:06:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:06:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:06:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:06:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:06:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:06:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:06:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:06:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:06:34 [DEBUG   ] Finished updating board flags in 1.772 seconds
2015-11-09 22:06:51 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 22:07:11 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-09 22:07:31 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 22:07:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:07:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:07:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:07:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:07:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:07:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:07:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:07:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:07:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:07:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:07:34 [DEBUG   ] Finished updating board flags in 1.618 seconds
2015-11-09 22:07:48 [DEBUG   ] Got command RPT from MCS: ref #531084
2015-11-09 22:07:48 [DEBUG   ] summary = ERROR
2015-11-09 22:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531084  13 57335 79668655 A  ERRORERROR'
2015-11-09 22:07:48 [DEBUG   ] Got command RPT from MCS: ref #531085
2015-11-09 22:07:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531085  84 57335 79668655 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:07:51 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:08:11 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 22:08:31 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:08:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:08:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:08:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:08:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:08:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:08:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:08:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:08:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:08:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:08:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:08:34 [DEBUG   ] Finished updating board flags in 1.644 seconds
2015-11-09 22:08:51 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:09:11 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:09:31 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:09:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:09:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:09:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:09:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:09:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:09:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:09:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:09:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:09:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:09:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:09:34 [DEBUG   ] Finished updating board flags in 1.655 seconds
2015-11-09 22:09:51 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:10:11 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:10:16 [DEBUG   ] Got command RPT from MCS: ref #531104
2015-11-09 22:10:16 [DEBUG   ] summary = ERROR
2015-11-09 22:10:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531104  13 57335 79816411 A  ERRORERROR'
2015-11-09 22:10:16 [DEBUG   ] Got command RPT from MCS: ref #531105
2015-11-09 22:10:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:10:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531105  84 57335 79816615 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:10:16 [DEBUG   ] Got command RPT from MCS: ref #531106
2015-11-09 22:10:16 [DEBUG   ] summary = ERROR
2015-11-09 22:10:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531106  13 57335 79816819 A  ERRORERROR'
2015-11-09 22:10:17 [DEBUG   ] Got command RPT from MCS: ref #531107
2015-11-09 22:10:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:10:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531107  84 57335 79817011 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:10:17 [DEBUG   ] Got command RPT from MCS: ref #531108
2015-11-09 22:10:17 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 22:10:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531108  12 57335 79817215 A  ERRORB�ÌÍ'
2015-11-09 22:10:31 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:10:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:10:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:10:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:10:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:10:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:10:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:10:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:10:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:10:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:10:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:10:34 [DEBUG   ] Finished updating board flags in 1.493 seconds
2015-11-09 22:10:51 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:11:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:11:31 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 22:11:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:11:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:11:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:11:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:11:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:11:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:11:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:11:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:11:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:11:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:11:34 [DEBUG   ] Finished updating board flags in 1.505 seconds
2015-11-09 22:11:51 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:12:11 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:12:31 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 22:12:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:12:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:12:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:12:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:12:33 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:12:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:12:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:12:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:12:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:12:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:12:34 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 22:12:34 [DEBUG   ] Finished updating board flags in 1.516 seconds
2015-11-09 22:12:48 [DEBUG   ] Got command RPT from MCS: ref #531112
2015-11-09 22:12:48 [DEBUG   ] summary = ERROR
2015-11-09 22:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531112  13 57335 79968655 A  ERRORERROR'
2015-11-09 22:12:48 [DEBUG   ] Got command RPT from MCS: ref #531113
2015-11-09 22:12:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531113  84 57335 79968655 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:12:51 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 22:13:11 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:13:31 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 22:13:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:13:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:13:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:13:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:13:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:13:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:13:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:13:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:13:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:13:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:13:35 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 22:13:35 [DEBUG   ] Finished updating board flags in 2.214 seconds
2015-11-09 22:13:51 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:14:11 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 22:14:31 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:14:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:14:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:14:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:14:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:14:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:14:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:14:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:14:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:14:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:14:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:14:35 [DEBUG   ] Finished updating board flags in 2.303 seconds
2015-11-09 22:14:51 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:15:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:15:16 [DEBUG   ] Got command RPT from MCS: ref #531133
2015-11-09 22:15:16 [DEBUG   ] summary = ERROR
2015-11-09 22:15:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531133  13 57335 80116515 A  ERRORERROR'
2015-11-09 22:15:16 [DEBUG   ] Got command RPT from MCS: ref #531134
2015-11-09 22:15:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:15:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531134  84 57335 80116719 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:15:16 [DEBUG   ] Got command RPT from MCS: ref #531135
2015-11-09 22:15:16 [DEBUG   ] summary = ERROR
2015-11-09 22:15:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531135  13 57335 80116923 A  ERRORERROR'
2015-11-09 22:15:17 [DEBUG   ] Got command RPT from MCS: ref #531136
2015-11-09 22:15:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:15:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531136  84 57335 80117127 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:15:17 [DEBUG   ] Got command RPT from MCS: ref #531137
2015-11-09 22:15:17 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 22:15:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531137  12 57335 80117319 A  ERRORB‹ÌÍ'
2015-11-09 22:15:31 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 22:15:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:15:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:15:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:15:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:15:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:15:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:15:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:15:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:15:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:15:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:15:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 22:15:35 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 22:15:35 [DEBUG   ] Finished updating board flags in 2.085 seconds
2015-11-09 22:15:51 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 22:16:11 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:16:11 [DEBUG   ] Got command TBW from MCS: ref #531144
2015-11-09 22:16:11 [DEBUG   ] tbw = exited with status True, code 0x00
2015-11-09 22:16:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBW   531144   8 57335 80171983 A  ERROR'
2015-11-09 22:16:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 pkill tbn
2015-11-09 22:16:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 pkill tbn
2015-11-09 22:16:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 pkill tbn
2015-11-09 22:16:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 pkill tbn
2015-11-09 22:16:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 pkill tbn
2015-11-09 22:16:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 pkill tbn
2015-11-09 22:16:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 pkill tbn
2015-11-09 22:16:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 pkill tbn
2015-11-09 22:16:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 pkill tbn
2015-11-09 22:16:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 pkill tbn
2015-11-09 22:16:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 pkill tbn
2015-11-09 22:16:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 pkill tbn
2015-11-09 22:16:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 pkill tbn
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa27', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '1', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa24', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '2', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa23', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '3', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa18', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '4', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa10', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '5', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa32', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '6', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa11', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '7', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa30', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '8', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa12', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '9', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa8', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '10', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa13', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '11', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa26', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '12', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa14', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '13', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa22', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '14', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa20', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '15', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa1', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '16', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa5', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '17', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa25', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '18', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa35', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '19', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa19', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '20', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa21', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '21', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa6', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '22', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa7', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '23', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa33', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '24', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa15', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '25', '12000000']
2015-11-09 22:16:15 [DEBUG   ] spawn_process: at 2015-11-09 22:16:15 running ['/usr/local/bin/lwa28', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '26', '12000000']
2015-11-09 22:16:15 [INFO    ] Finished spawning TBW
2015-11-09 22:16:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:16:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:16:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:16:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:16:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:16:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:16:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:16:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:16:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:16:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:16:35 [DEBUG   ] Finished updating board flags in 1.800 seconds
2015-11-09 22:17:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:17:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:17:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:17:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:17:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:17:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:17:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:17:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:17:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:17:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:17:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 22:17:35 [DEBUG   ] Finished updating board flags in 1.881 seconds
2015-11-09 22:17:48 [DEBUG   ] Got command RPT from MCS: ref #531152
2015-11-09 22:17:48 [DEBUG   ] summary = ERROR
2015-11-09 22:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531152  13 57335 80268659 A  ERRORERROR'
2015-11-09 22:17:48 [DEBUG   ] Got command RPT from MCS: ref #531153
2015-11-09 22:17:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531153  84 57335 80268659 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:18:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:18:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:18:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:18:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:18:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:18:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:18:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:18:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:18:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:18:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:18:35 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 22:18:35 [DEBUG   ] Finished updating board flags in 1.742 seconds
2015-11-09 22:19:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:19:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:19:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:19:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:19:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:19:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:19:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:19:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:19:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:19:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:19:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 22:19:35 [DEBUG   ] Finished updating board flags in 1.685 seconds
2015-11-09 22:19:53 [INFO    ] __waitForTBW - TBW finished in 218.277 s
2015-11-09 22:19:54 [DEBUG   ] Finished updating ant stats in 0.487 seconds
2015-11-09 22:19:54 [INFO    ] Started antennas statistics background thread
2015-11-09 22:19:54 [INFO    ] Resuming TBN at 38.100 MHz after TBW acquisition
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '1', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '2', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '3', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '4', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '5', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '6', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '7', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '8', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '9', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '10', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '11', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '12', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '13', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '14', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '15', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '16', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '17', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '18', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '19', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '20', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '21', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '22', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '23', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '24', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '25', '1447107596']
2015-11-09 22:19:54 [DEBUG   ] spawn_process: at 2015-11-09 22:19:54 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '26', '1447107596']
2015-11-09 22:19:54 [INFO    ] Finished spawning TBN in 0.057 s
2015-11-09 22:20:14 [DEBUG   ] Finished updating ant stats in 0.470 seconds
2015-11-09 22:20:16 [DEBUG   ] Got command RPT from MCS: ref #531174
2015-11-09 22:20:16 [DEBUG   ] summary = ERROR
2015-11-09 22:20:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531174  13 57335 80416651 A  ERRORERROR'
2015-11-09 22:20:16 [DEBUG   ] Got command RPT from MCS: ref #531175
2015-11-09 22:20:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:20:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531175  84 57335 80416855 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:20:17 [DEBUG   ] Got command RPT from MCS: ref #531176
2015-11-09 22:20:17 [DEBUG   ] summary = ERROR
2015-11-09 22:20:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531176  13 57335 80417083 A  ERRORERROR'
2015-11-09 22:20:17 [DEBUG   ] Got command RPT from MCS: ref #531177
2015-11-09 22:20:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:20:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531177  84 57335 80417255 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:20:17 [DEBUG   ] Got command RPT from MCS: ref #531178
2015-11-09 22:20:17 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 22:20:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531178  12 57335 80417459 A  ERRORBŠÌÍ'
2015-11-09 22:20:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:20:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:20:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:20:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:20:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:20:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:20:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:20:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:20:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:20:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:20:35 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-09 22:20:35 [DEBUG   ] Finished updating board flags in 1.824 seconds
2015-11-09 22:20:55 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 22:21:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:21:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:21:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:21:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:21:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:21:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:21:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:21:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:21:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:21:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:21:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:21:35 [DEBUG   ] Finished updating ant stats in 1.258 seconds
2015-11-09 22:21:35 [DEBUG   ] Finished updating board flags in 1.838 seconds
2015-11-09 22:21:55 [DEBUG   ] Finished updating ant stats in 1.148 seconds
2015-11-09 22:22:15 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 22:22:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:22:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:22:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:22:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:22:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:22:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:22:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:22:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:22:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:22:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:22:35 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 22:22:35 [DEBUG   ] Finished updating ant stats in 1.045 seconds
2015-11-09 22:22:35 [DEBUG   ] Finished updating board flags in 1.672 seconds
2015-11-09 22:22:48 [DEBUG   ] Got command RPT from MCS: ref #531185
2015-11-09 22:22:48 [DEBUG   ] summary = ERROR
2015-11-09 22:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531185  13 57335 80568659 A  ERRORERROR'
2015-11-09 22:22:48 [DEBUG   ] Got command RPT from MCS: ref #531186
2015-11-09 22:22:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531186  84 57335 80568659 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:22:55 [DEBUG   ] Finished updating ant stats in 0.959 seconds
2015-11-09 22:23:15 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:23:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:23:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:23:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:23:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:23:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:23:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:23:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:23:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:23:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:23:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:23:35 [DEBUG   ] Finished updating ant stats in 0.846 seconds
2015-11-09 22:23:35 [DEBUG   ] Finished updating board flags in 1.615 seconds
2015-11-09 22:23:55 [DEBUG   ] Finished updating ant stats in 0.757 seconds
2015-11-09 22:24:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 22:24:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:24:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:24:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:24:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:24:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:24:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:24:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:24:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:24:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:24:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:24:35 [DEBUG   ] Finished updating ant stats in 0.644 seconds
2015-11-09 22:24:35 [DEBUG   ] Finished updating board flags in 1.610 seconds
2015-11-09 22:24:55 [DEBUG   ] Finished updating ant stats in 0.559 seconds
2015-11-09 22:25:16 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 22:25:16 [DEBUG   ] Got command RPT from MCS: ref #531206
2015-11-09 22:25:16 [DEBUG   ] summary = ERROR
2015-11-09 22:25:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531206  13 57335 80716751 A  ERRORERROR'
2015-11-09 22:25:16 [DEBUG   ] Got command RPT from MCS: ref #531207
2015-11-09 22:25:16 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:25:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531207  84 57335 80716955 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:25:17 [DEBUG   ] Got command RPT from MCS: ref #531208
2015-11-09 22:25:17 [DEBUG   ] summary = ERROR
2015-11-09 22:25:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531208  13 57335 80717159 A  ERRORERROR'
2015-11-09 22:25:17 [DEBUG   ] Got command RPT from MCS: ref #531209
2015-11-09 22:25:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:25:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531209  84 57335 80717351 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:25:17 [DEBUG   ] Got command RPT from MCS: ref #531210
2015-11-09 22:25:17 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 22:25:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531210  12 57335 80717555 A  ERRORB�ÌÍ'
2015-11-09 22:25:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:25:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:25:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:25:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:25:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:25:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:25:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:25:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:25:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:25:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:25:35 [DEBUG   ] Finished updating board flags in 1.552 seconds
2015-11-09 22:25:36 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 22:25:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:26:16 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 22:26:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:26:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:26:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:26:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:26:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:26:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:26:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:26:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:26:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:26:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:26:35 [DEBUG   ] Finished updating board flags in 1.468 seconds
2015-11-09 22:26:36 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 22:26:56 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 22:27:16 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-09 22:27:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:27:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:27:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:27:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:27:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:27:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:27:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:27:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:27:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:27:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:27:35 [DEBUG   ] Finished updating board flags in 1.449 seconds
2015-11-09 22:27:36 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:27:48 [DEBUG   ] Got command RPT from MCS: ref #531219
2015-11-09 22:27:48 [DEBUG   ] summary = ERROR
2015-11-09 22:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531219  13 57335 80868655 A  ERRORERROR'
2015-11-09 22:27:48 [DEBUG   ] Got command RPT from MCS: ref #531220
2015-11-09 22:27:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531220  84 57335 80868655 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:27:56 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:28:16 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 22:28:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:28:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:28:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:28:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:28:34 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:28:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:28:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:28:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:28:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:28:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:28:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 22:28:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 22:28:35 [DEBUG   ] Finished updating board flags in 1.453 seconds
2015-11-09 22:28:36 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:28:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 22:29:16 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:29:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:29:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:29:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:29:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:29:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:29:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:29:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:29:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:29:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:29:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:29:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 22:29:36 [DEBUG   ] Finished updating ant stats in 0.847 seconds
2015-11-09 22:29:36 [DEBUG   ] Finished updating board flags in 2.274 seconds
2015-11-09 22:29:56 [DEBUG   ] Finished updating ant stats in 0.758 seconds
2015-11-09 22:30:16 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 22:30:16 [DEBUG   ] Got command RPT from MCS: ref #531239
2015-11-09 22:30:16 [DEBUG   ] summary = ERROR
2015-11-09 22:30:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531239  13 57335 81016863 A  ERRORERROR'
2015-11-09 22:30:17 [DEBUG   ] Got command RPT from MCS: ref #531240
2015-11-09 22:30:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:30:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531240  84 57335 81017067 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:30:17 [DEBUG   ] Got command RPT from MCS: ref #531241
2015-11-09 22:30:17 [DEBUG   ] summary = ERROR
2015-11-09 22:30:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531241  13 57335 81017271 A  ERRORERROR'
2015-11-09 22:30:17 [DEBUG   ] Got command RPT from MCS: ref #531242
2015-11-09 22:30:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:30:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531242  84 57335 81017475 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:30:17 [DEBUG   ] Got command RPT from MCS: ref #531243
2015-11-09 22:30:17 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 22:30:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531243  12 57335 81017667 A  ERRORBŒÌÍ'
2015-11-09 22:30:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:30:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:30:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:30:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:30:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:30:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:30:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:30:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:30:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:30:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:30:36 [DEBUG   ] Finished updating ant stats in 0.644 seconds
2015-11-09 22:30:36 [DEBUG   ] Finished updating board flags in 2.187 seconds
2015-11-09 22:30:56 [DEBUG   ] Finished updating ant stats in 0.559 seconds
2015-11-09 22:31:17 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2015-11-09 22:31:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:31:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:31:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:31:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:31:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:31:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:31:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:31:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:31:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:31:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:31:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 22:31:36 [DEBUG   ] Finished updating board flags in 1.990 seconds
2015-11-09 22:31:37 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 22:31:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:32:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:32:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:32:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:32:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:32:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:32:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:32:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:32:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:32:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:32:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:32:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:32:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 22:32:36 [DEBUG   ] Finished updating board flags in 1.807 seconds
2015-11-09 22:32:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:32:48 [DEBUG   ] Got command RPT from MCS: ref #531247
2015-11-09 22:32:48 [DEBUG   ] summary = ERROR
2015-11-09 22:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531247  13 57335 81168659 A  ERRORERROR'
2015-11-09 22:32:48 [DEBUG   ] Got command RPT from MCS: ref #531248
2015-11-09 22:32:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531248  84 57335 81168659 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:32:57 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 22:33:17 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 22:33:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:33:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:33:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:33:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:33:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:33:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:33:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:33:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:33:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:33:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:33:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 22:33:36 [DEBUG   ] Finished updating board flags in 1.835 seconds
2015-11-09 22:33:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:33:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:34:17 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 22:34:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:34:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:34:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:34:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:34:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:34:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:34:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:34:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:34:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:34:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:34:36 [DEBUG   ] Finished updating board flags in 1.797 seconds
2015-11-09 22:34:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:34:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:35:16 [DEBUG   ] Got command RPT from MCS: ref #531265
2015-11-09 22:35:16 [DEBUG   ] summary = ERROR
2015-11-09 22:35:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531265  13 57335 81316943 A  ERRORERROR'
2015-11-09 22:35:17 [DEBUG   ] Got command RPT from MCS: ref #531266
2015-11-09 22:35:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:35:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531266  84 57335 81317147 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:35:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:35:17 [DEBUG   ] Got command RPT from MCS: ref #531267
2015-11-09 22:35:17 [DEBUG   ] summary = ERROR
2015-11-09 22:35:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531267  13 57335 81317351 A  ERRORERROR'
2015-11-09 22:35:17 [DEBUG   ] Got command RPT from MCS: ref #531268
2015-11-09 22:35:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:35:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531268  84 57335 81317555 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:35:17 [DEBUG   ] Got command RPT from MCS: ref #531269
2015-11-09 22:35:17 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 22:35:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531269  12 57335 81317747 A  ERRORB‹ÌÍ'
2015-11-09 22:35:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:35:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:35:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:35:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:35:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:35:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:35:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:35:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:35:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:35:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:35:36 [DEBUG   ] Finished updating board flags in 1.613 seconds
2015-11-09 22:35:37 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 22:35:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:36:17 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-09 22:36:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:36:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:36:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:36:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:36:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:36:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 22:36:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:36:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:36:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:36:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:36:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:36:36 [DEBUG   ] Finished updating board flags in 1.551 seconds
2015-11-09 22:36:37 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 22:36:57 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 22:37:17 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-09 22:37:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:37:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:37:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:37:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:37:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:37:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:37:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:37:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:37:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:37:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:37:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 22:37:36 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 22:37:36 [DEBUG   ] Finished updating board flags in 1.514 seconds
2015-11-09 22:37:37 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:37:48 [DEBUG   ] Got command RPT from MCS: ref #531278
2015-11-09 22:37:48 [DEBUG   ] summary = ERROR
2015-11-09 22:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531278  13 57335 81468655 A  ERRORERROR'
2015-11-09 22:37:48 [DEBUG   ] Got command RPT from MCS: ref #531279
2015-11-09 22:37:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531279  84 57335 81468655 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:37:57 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:38:17 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 22:38:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:38:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:38:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:38:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:38:35 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:38:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:38:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:38:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:38:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:38:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:38:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 22:38:36 [DEBUG   ] Finished updating board flags in 1.511 seconds
2015-11-09 22:38:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 22:38:57 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:39:17 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:39:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:39:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:39:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:39:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:39:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:39:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:39:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:39:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:39:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:39:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:39:37 [DEBUG   ] Finished updating ant stats in 0.847 seconds
2015-11-09 22:39:37 [DEBUG   ] Finished updating board flags in 2.218 seconds
2015-11-09 22:39:57 [DEBUG   ] Finished updating ant stats in 0.758 seconds
2015-11-09 22:40:17 [DEBUG   ] Got command RPT from MCS: ref #531298
2015-11-09 22:40:17 [DEBUG   ] summary = ERROR
2015-11-09 22:40:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531298  13 57335 81617083 A  ERRORERROR'
2015-11-09 22:40:17 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 22:40:17 [DEBUG   ] Got command RPT from MCS: ref #531299
2015-11-09 22:40:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:40:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531299  84 57335 81617479 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:40:17 [DEBUG   ] Got command RPT from MCS: ref #531300
2015-11-09 22:40:17 [DEBUG   ] summary = ERROR
2015-11-09 22:40:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531300  13 57335 81617683 A  ERRORERROR'
2015-11-09 22:40:17 [DEBUG   ] Got command RPT from MCS: ref #531301
2015-11-09 22:40:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:40:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531301  84 57335 81617887 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:40:18 [DEBUG   ] Got command RPT from MCS: ref #531302
2015-11-09 22:40:18 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 22:40:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531302  12 57335 81618091 A  ERRORBŠÌÍ'
2015-11-09 22:40:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:40:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:40:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:40:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:40:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:40:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:40:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:40:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:40:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:40:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:40:37 [DEBUG   ] Finished updating ant stats in 0.645 seconds
2015-11-09 22:40:37 [DEBUG   ] Finished updating board flags in 2.198 seconds
2015-11-09 22:40:57 [DEBUG   ] Finished updating ant stats in 0.560 seconds
2015-11-09 22:41:18 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-09 22:41:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:41:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:41:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:41:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:41:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:41:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:41:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:41:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:41:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:41:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:41:37 [DEBUG   ] Finished updating board flags in 1.970 seconds
2015-11-09 22:41:38 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 22:41:58 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 22:42:18 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:42:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:42:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:42:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:42:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:42:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:42:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:42:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:42:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:42:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:42:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:42:37 [DEBUG   ] Finished updating board flags in 1.827 seconds
2015-11-09 22:42:38 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:42:48 [DEBUG   ] Got command RPT from MCS: ref #531306
2015-11-09 22:42:48 [DEBUG   ] summary = ERROR
2015-11-09 22:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531306  13 57335 81768659 A  ERRORERROR'
2015-11-09 22:42:48 [DEBUG   ] Got command RPT from MCS: ref #531307
2015-11-09 22:42:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531307  84 57335 81768659 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:42:58 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:43:18 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 22:43:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:43:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:43:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:43:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:43:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:43:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:43:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:43:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:43:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:43:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:43:37 [DEBUG   ] Finished updating board flags in 1.773 seconds
2015-11-09 22:43:38 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:43:58 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 22:44:18 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 22:44:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:44:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:44:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:44:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:44:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:44:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:44:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:44:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:44:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:44:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:44:37 [DEBUG   ] Finished updating board flags in 1.628 seconds
2015-11-09 22:44:38 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 22:44:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 22:45:17 [DEBUG   ] Got command RPT from MCS: ref #531327
2015-11-09 22:45:17 [DEBUG   ] summary = ERROR
2015-11-09 22:45:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531327  13 57335 81917179 A  ERRORERROR'
2015-11-09 22:45:17 [DEBUG   ] Got command RPT from MCS: ref #531328
2015-11-09 22:45:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:45:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531328  84 57335 81917383 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:45:17 [DEBUG   ] Got command RPT from MCS: ref #531329
2015-11-09 22:45:17 [DEBUG   ] summary = ERROR
2015-11-09 22:45:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531329  13 57335 81917575 A  ERRORERROR'
2015-11-09 22:45:17 [DEBUG   ] Got command RPT from MCS: ref #531330
2015-11-09 22:45:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:45:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531330  84 57335 81917779 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:45:17 [DEBUG   ] Got command RPT from MCS: ref #531331
2015-11-09 22:45:17 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 22:45:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531331  12 57335 81917983 A  ERRORBŒÌÍ'
2015-11-09 22:45:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 22:45:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:45:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:45:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:45:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:45:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:45:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:45:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:45:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:45:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:45:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:45:37 [DEBUG   ] Finished updating board flags in 1.642 seconds
2015-11-09 22:45:38 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 22:45:58 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 22:46:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 22:46:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:46:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:46:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:46:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:46:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:46:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:46:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:46:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:46:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:46:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:46:37 [DEBUG   ] Finished updating board flags in 1.574 seconds
2015-11-09 22:46:38 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 22:46:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 22:47:18 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 22:47:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:47:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:47:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:47:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:47:36 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:47:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:47:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:47:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:47:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:47:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:47:37 [DEBUG   ] Finished updating board flags in 1.592 seconds
2015-11-09 22:47:38 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 22:47:48 [DEBUG   ] Got command RPT from MCS: ref #531340
2015-11-09 22:47:48 [DEBUG   ] summary = ERROR
2015-11-09 22:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531340  13 57335 82068663 A  ERRORERROR'
2015-11-09 22:47:48 [DEBUG   ] Got command RPT from MCS: ref #531341
2015-11-09 22:47:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531341  84 57335 82068663 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:47:58 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:48:18 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:48:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:48:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:48:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:48:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:48:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:48:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:48:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:48:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:48:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:48:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:48:38 [DEBUG   ] Finished updating ant stats in 0.847 seconds
2015-11-09 22:48:38 [DEBUG   ] Finished updating board flags in 2.253 seconds
2015-11-09 22:48:58 [DEBUG   ] Finished updating ant stats in 0.758 seconds
2015-11-09 22:49:18 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 22:49:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:49:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:49:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:49:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:49:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:49:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:49:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:49:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:49:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:49:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:49:38 [DEBUG   ] Finished updating ant stats in 0.644 seconds
2015-11-09 22:49:38 [DEBUG   ] Finished updating board flags in 2.139 seconds
2015-11-09 22:49:58 [DEBUG   ] Finished updating ant stats in 0.559 seconds
2015-11-09 22:50:17 [DEBUG   ] Got command RPT from MCS: ref #531357
2015-11-09 22:50:17 [DEBUG   ] summary = ERROR
2015-11-09 22:50:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531357  13 57335 82217291 A  ERRORERROR'
2015-11-09 22:50:17 [DEBUG   ] Got command RPT from MCS: ref #531358
2015-11-09 22:50:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:50:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531358  84 57335 82217483 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:50:17 [DEBUG   ] Got command RPT from MCS: ref #531359
2015-11-09 22:50:17 [DEBUG   ] summary = ERROR
2015-11-09 22:50:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531359  13 57335 82217690 A  ERRORERROR'
2015-11-09 22:50:17 [DEBUG   ] Got command RPT from MCS: ref #531360
2015-11-09 22:50:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:50:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531360  84 57335 82217891 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:50:18 [DEBUG   ] Got command RPT from MCS: ref #531361
2015-11-09 22:50:18 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 22:50:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531361  12 57335 82218095 A  ERRORBŒÌÍ'
2015-11-09 22:50:19 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-09 22:50:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:50:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:50:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:50:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:50:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:50:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:50:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:50:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:50:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:50:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:50:38 [DEBUG   ] Finished updating board flags in 2.015 seconds
2015-11-09 22:50:39 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 22:50:59 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 22:51:19 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 22:51:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:51:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:51:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:51:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:51:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:51:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:51:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:51:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:51:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:51:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:51:38 [DEBUG   ] Finished updating board flags in 2.006 seconds
2015-11-09 22:51:39 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 22:51:59 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 22:52:19 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 22:52:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:52:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:52:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:52:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:52:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:52:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:52:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:52:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:52:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:52:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:52:38 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 22:52:38 [DEBUG   ] Finished updating board flags in 1.981 seconds
2015-11-09 22:52:39 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 22:52:48 [DEBUG   ] Got command RPT from MCS: ref #531365
2015-11-09 22:52:48 [DEBUG   ] summary = ERROR
2015-11-09 22:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531365  13 57335 82368667 A  ERRORERROR'
2015-11-09 22:52:48 [DEBUG   ] Got command RPT from MCS: ref #531366
2015-11-09 22:52:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531366  84 57335 82368667 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:52:59 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-09 22:53:19 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-09 22:53:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:53:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:53:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:53:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:53:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:53:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:53:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:53:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:53:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:53:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:53:38 [DEBUG   ] Finished updating board flags in 1.797 seconds
2015-11-09 22:53:39 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-09 22:53:59 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:54:19 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:54:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:54:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:54:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:54:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:54:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:54:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:54:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:54:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:54:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:54:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:54:38 [DEBUG   ] Finished updating board flags in 1.601 seconds
2015-11-09 22:54:39 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 22:54:59 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:55:17 [DEBUG   ] Got command RPT from MCS: ref #531386
2015-11-09 22:55:17 [DEBUG   ] summary = ERROR
2015-11-09 22:55:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531386  13 57335 82517379 A  ERRORERROR'
2015-11-09 22:55:17 [DEBUG   ] Got command RPT from MCS: ref #531387
2015-11-09 22:55:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:55:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531387  84 57335 82517583 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:55:17 [DEBUG   ] Got command RPT from MCS: ref #531388
2015-11-09 22:55:17 [DEBUG   ] summary = ERROR
2015-11-09 22:55:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531388  13 57335 82517775 A  ERRORERROR'
2015-11-09 22:55:17 [DEBUG   ] Got command RPT from MCS: ref #531389
2015-11-09 22:55:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:55:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531389  84 57335 82517983 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:55:18 [DEBUG   ] Got command RPT from MCS: ref #531390
2015-11-09 22:55:18 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 22:55:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531390  12 57335 82518175 A  ERRORB‹ÌÍ'
2015-11-09 22:55:19 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 22:55:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:55:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:55:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:55:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:55:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:55:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:55:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:55:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:55:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:55:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:55:38 [DEBUG   ] Finished updating board flags in 1.617 seconds
2015-11-09 22:55:39 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:55:59 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 22:56:19 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2015-11-09 22:56:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:56:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:56:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:56:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:56:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:56:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:56:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:56:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:56:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:56:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:56:38 [DEBUG   ] Finished updating board flags in 1.540 seconds
2015-11-09 22:56:39 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 22:57:00 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 22:57:20 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-09 22:57:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:57:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:57:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:57:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:57:37 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:57:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:57:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:57:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:57:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:57:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:57:38 [DEBUG   ] Finished updating board flags in 1.397 seconds
2015-11-09 22:57:40 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-09 22:57:48 [DEBUG   ] Got command RPT from MCS: ref #531399
2015-11-09 22:57:48 [DEBUG   ] summary = ERROR
2015-11-09 22:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531399  13 57335 82668667 A  ERRORERROR'
2015-11-09 22:57:48 [DEBUG   ] Got command RPT from MCS: ref #531400
2015-11-09 22:57:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 22:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531400  84 57335 82668667 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 22:58:00 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 22:58:20 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 22:58:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998529
2015-11-09 22:58:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998529
2015-11-09 22:58:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998529
2015-11-09 22:58:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998529
2015-11-09 22:58:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998529
2015-11-09 22:58:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998529
2015-11-09 22:58:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998529
2015-11-09 22:58:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998529
2015-11-09 22:58:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998529
2015-11-09 22:58:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998529
2015-11-09 22:58:39 [DEBUG   ] Finished updating board flags in 2.250 seconds
2015-11-09 22:58:40 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 22:59:00 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 22:59:20 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-09 22:59:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-09 22:59:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-09 22:59:39 [DEBUG   ] Finished updating board flags in 2.297 seconds
2015-11-09 22:59:40 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 23:00:00 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 23:00:17 [DEBUG   ] Got command RPT from MCS: ref #531419
2015-11-09 23:00:17 [DEBUG   ] summary = ERROR
2015-11-09 23:00:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531419  13 57335 82817467 A  ERRORERROR'
2015-11-09 23:00:17 [DEBUG   ] Got command RPT from MCS: ref #531420
2015-11-09 23:00:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:00:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531420  84 57335 82817671 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:00:17 [DEBUG   ] Got command RPT from MCS: ref #531421
2015-11-09 23:00:17 [DEBUG   ] summary = ERROR
2015-11-09 23:00:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531421  13 57335 82817875 A  ERRORERROR'
2015-11-09 23:00:18 [DEBUG   ] Got command RPT from MCS: ref #531422
2015-11-09 23:00:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:00:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531422  84 57335 82818067 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:00:18 [DEBUG   ] Got command RPT from MCS: ref #531423
2015-11-09 23:00:18 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 23:00:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531423  12 57335 82818271 A  ERRORBŠÌÍ'
2015-11-09 23:00:20 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 23:00:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 23:00:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 23:00:39 [DEBUG   ] Finished updating board flags in 2.120 seconds
2015-11-09 23:00:40 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:01:00 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 23:01:20 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 23:01:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:01:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:01:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:01:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:01:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:01:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:01:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:01:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:01:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:01:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:01:39 [DEBUG   ] Finished updating board flags in 1.994 seconds
2015-11-09 23:01:40 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:02:00 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:02:20 [DEBUG   ] Finished updating ant stats in 0.938 seconds
2015-11-09 23:02:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:02:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:02:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:02:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:02:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:02:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:02:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:02:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:02:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:02:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:02:39 [DEBUG   ] Finished updating board flags in 1.764 seconds
2015-11-09 23:02:40 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 23:02:48 [DEBUG   ] Got command RPT from MCS: ref #531427
2015-11-09 23:02:48 [DEBUG   ] summary = ERROR
2015-11-09 23:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531427  13 57335 82968659 A  ERRORERROR'
2015-11-09 23:02:48 [DEBUG   ] Got command RPT from MCS: ref #531428
2015-11-09 23:02:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531428  84 57335 82968659 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:03:00 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-09 23:03:20 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 23:03:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:03:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:03:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:03:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:03:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:03:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:03:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:03:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:03:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:03:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:03:39 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 23:03:39 [DEBUG   ] Finished updating board flags in 1.595 seconds
2015-11-09 23:03:40 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 23:04:00 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 23:04:20 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 23:04:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:04:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:04:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:04:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:04:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:04:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:04:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:04:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:04:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:04:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:04:39 [DEBUG   ] Finished updating board flags in 1.424 seconds
2015-11-09 23:04:40 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 23:05:00 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 23:05:17 [DEBUG   ] Got command RPT from MCS: ref #531445
2015-11-09 23:05:17 [DEBUG   ] summary = ERROR
2015-11-09 23:05:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531445  13 57335 83117563 A  ERRORERROR'
2015-11-09 23:05:17 [DEBUG   ] Got command RPT from MCS: ref #531446
2015-11-09 23:05:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:05:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531446  84 57335 83117767 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:05:17 [DEBUG   ] Got command RPT from MCS: ref #531447
2015-11-09 23:05:17 [DEBUG   ] summary = ERROR
2015-11-09 23:05:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531447  13 57335 83117959 A  ERRORERROR'
2015-11-09 23:05:18 [DEBUG   ] Got command RPT from MCS: ref #531448
2015-11-09 23:05:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:05:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531448  84 57335 83118163 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:05:18 [DEBUG   ] Got command RPT from MCS: ref #531449
2015-11-09 23:05:18 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 23:05:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531449  12 57335 83118367 A  ERRORB�ÌÍ'
2015-11-09 23:05:20 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 23:05:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:05:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:05:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:05:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:05:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:05:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:05:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:05:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:05:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:05:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:05:39 [DEBUG   ] Finished updating board flags in 1.426 seconds
2015-11-09 23:05:40 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 23:06:00 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 23:06:20 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 23:06:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:06:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:06:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:06:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:06:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:06:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:06:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:06:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:06:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:06:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:06:39 [DEBUG   ] Finished updating board flags in 1.603 seconds
2015-11-09 23:06:40 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 23:07:00 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-09 23:07:21 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 23:07:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:07:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:07:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:07:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:07:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:07:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:07:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:07:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:07:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:07:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:07:39 [DEBUG   ] Finished updating board flags in 1.338 seconds
2015-11-09 23:07:41 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 23:07:48 [DEBUG   ] Got command RPT from MCS: ref #531458
2015-11-09 23:07:48 [DEBUG   ] summary = ERROR
2015-11-09 23:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531458  13 57335 83268667 A  ERRORERROR'
2015-11-09 23:07:48 [DEBUG   ] Got command RPT from MCS: ref #531459
2015-11-09 23:07:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531459  84 57335 83268667 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:08:01 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 23:08:21 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 23:08:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:08:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:08:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:08:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:08:38 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:08:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:08:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:08:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:08:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:08:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:08:39 [DEBUG   ] Finished updating board flags in 1.374 seconds
2015-11-09 23:08:41 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 23:09:01 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-09 23:09:21 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-09 23:09:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:09:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:09:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:09:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:09:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:09:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:09:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:09:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:09:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:09:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:09:40 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 23:09:40 [DEBUG   ] Finished updating board flags in 2.092 seconds
2015-11-09 23:09:41 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:10:01 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 23:10:17 [DEBUG   ] Got command RPT from MCS: ref #531478
2015-11-09 23:10:17 [DEBUG   ] summary = ERROR
2015-11-09 23:10:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531478  13 57335 83417639 A  ERRORERROR'
2015-11-09 23:10:17 [DEBUG   ] Got command RPT from MCS: ref #531479
2015-11-09 23:10:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:10:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531479  84 57335 83417843 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:10:18 [DEBUG   ] Got command RPT from MCS: ref #531480
2015-11-09 23:10:18 [DEBUG   ] summary = ERROR
2015-11-09 23:10:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531480  13 57335 83418047 A  ERRORERROR'
2015-11-09 23:10:18 [DEBUG   ] Got command RPT from MCS: ref #531481
2015-11-09 23:10:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:10:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531481  84 57335 83418239 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:10:18 [DEBUG   ] Got command RPT from MCS: ref #531482
2015-11-09 23:10:18 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 23:10:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531482  12 57335 83418443 A  ERRORBŒÌÍ'
2015-11-09 23:10:21 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-09 23:10:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:10:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:10:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:10:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:10:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:10:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:10:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:10:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:10:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:10:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:10:40 [DEBUG   ] Finished updating board flags in 1.968 seconds
2015-11-09 23:10:41 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-09 23:11:01 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-09 23:11:21 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-09 23:11:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:11:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:11:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:11:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:11:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:11:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:11:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:11:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:11:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:11:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:11:40 [DEBUG   ] Finished updating board flags in 1.800 seconds
2015-11-09 23:11:42 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2015-11-09 23:12:02 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 23:12:22 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 23:12:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:12:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:12:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:12:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:12:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:12:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:12:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:12:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:12:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:12:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:12:40 [DEBUG   ] Finished updating board flags in 1.582 seconds
2015-11-09 23:12:42 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 23:12:48 [DEBUG   ] Got command RPT from MCS: ref #531486
2015-11-09 23:12:48 [DEBUG   ] summary = ERROR
2015-11-09 23:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531486  13 57335 83568663 A  ERRORERROR'
2015-11-09 23:12:48 [DEBUG   ] Got command RPT from MCS: ref #531487
2015-11-09 23:12:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531487  84 57335 83568663 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:13:02 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 23:13:22 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 23:13:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:13:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:13:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:13:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:13:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:13:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:13:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:13:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:13:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:13:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:13:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 23:13:40 [DEBUG   ] Finished updating board flags in 1.424 seconds
2015-11-09 23:13:42 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 23:14:02 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 23:14:22 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 23:14:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:14:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:14:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:14:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:14:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:14:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:14:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:14:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:14:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:14:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:14:40 [DEBUG   ] Finished updating board flags in 1.424 seconds
2015-11-09 23:14:42 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 23:15:02 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 23:15:17 [DEBUG   ] Got command RPT from MCS: ref #531512
2015-11-09 23:15:17 [DEBUG   ] summary = ERROR
2015-11-09 23:15:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531512  13 57335 83717755 A  ERRORERROR'
2015-11-09 23:15:17 [DEBUG   ] Got command RPT from MCS: ref #531513
2015-11-09 23:15:17 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:15:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531513  84 57335 83717947 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:15:18 [DEBUG   ] Got command RPT from MCS: ref #531514
2015-11-09 23:15:18 [DEBUG   ] summary = ERROR
2015-11-09 23:15:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531514  13 57335 83718151 A  ERRORERROR'
2015-11-09 23:15:18 [DEBUG   ] Got command RPT from MCS: ref #531515
2015-11-09 23:15:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:15:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531515  84 57335 83718355 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:15:18 [DEBUG   ] Got command RPT from MCS: ref #531516
2015-11-09 23:15:18 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 23:15:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531516  12 57335 83718559 A  ERRORB‹ÌÍ'
2015-11-09 23:15:22 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 23:15:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:15:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:15:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:15:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:15:39 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:15:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:15:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:15:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:15:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:15:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:15:40 [DEBUG   ] Finished updating board flags in 1.554 seconds
2015-11-09 23:15:42 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 23:16:02 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 23:16:22 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 23:16:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:16:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:16:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:16:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:16:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:16:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:16:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:16:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:16:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:16:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:16:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 23:16:41 [DEBUG   ] Finished updating board flags in 2.288 seconds
2015-11-09 23:16:42 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 23:17:03 [DEBUG   ] Finished updating ant stats in 2.201 seconds
2015-11-09 23:17:22 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 23:17:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:17:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:17:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:17:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:17:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:17:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:17:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:17:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:17:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:17:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:17:41 [DEBUG   ] Finished updating board flags in 2.238 seconds
2015-11-09 23:17:42 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 23:17:48 [DEBUG   ] Got command RPT from MCS: ref #531525
2015-11-09 23:17:48 [DEBUG   ] summary = ERROR
2015-11-09 23:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531525  13 57335 83868663 A  ERRORERROR'
2015-11-09 23:17:48 [DEBUG   ] Got command RPT from MCS: ref #531526
2015-11-09 23:17:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531526  84 57335 83868663 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:18:02 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 23:18:22 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 23:18:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:18:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:18:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:18:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:18:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:18:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:18:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:18:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:18:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:18:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:18:41 [DEBUG   ] Finished updating board flags in 2.147 seconds
2015-11-09 23:18:42 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 23:19:02 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 23:19:22 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-09 23:19:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:19:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:19:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:19:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:19:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:19:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:19:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:19:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:19:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:19:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:19:41 [DEBUG   ] Finished updating board flags in 1.987 seconds
2015-11-09 23:19:42 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 23:20:02 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 23:20:17 [DEBUG   ] Got command RPT from MCS: ref #531542
2015-11-09 23:20:17 [DEBUG   ] summary = ERROR
2015-11-09 23:20:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531542  13 57335 84017859 A  ERRORERROR'
2015-11-09 23:20:18 [DEBUG   ] Got command RPT from MCS: ref #531543
2015-11-09 23:20:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:20:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531543  84 57335 84018063 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:20:18 [DEBUG   ] Got command RPT from MCS: ref #531544
2015-11-09 23:20:18 [DEBUG   ] summary = ERROR
2015-11-09 23:20:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531544  13 57335 84018255 A  ERRORERROR'
2015-11-09 23:20:18 [DEBUG   ] Got command RPT from MCS: ref #531545
2015-11-09 23:20:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:20:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531545  84 57335 84018667 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:20:18 [DEBUG   ] Got command RPT from MCS: ref #531546
2015-11-09 23:20:18 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 23:20:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531546  12 57335 84018859 A  ERRORB‹ÌÍ'
2015-11-09 23:20:22 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 23:20:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:20:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:20:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:20:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:20:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:20:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:20:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:20:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:20:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:20:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:20:42 [DEBUG   ] Finished updating board flags in 2.099 seconds
2015-11-09 23:20:42 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 23:21:02 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:21:22 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:21:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:21:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:21:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:21:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:21:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:21:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:21:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:21:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:21:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:21:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:21:41 [DEBUG   ] Finished updating board flags in 1.726 seconds
2015-11-09 23:21:42 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 23:22:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 23:22:22 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:22:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:22:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:22:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:22:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:22:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:22:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:22:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:22:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:22:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:22:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:22:41 [DEBUG   ] Finished updating board flags in 1.644 seconds
2015-11-09 23:22:42 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:22:48 [DEBUG   ] Got command RPT from MCS: ref #531550
2015-11-09 23:22:48 [DEBUG   ] summary = ERROR
2015-11-09 23:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531550  13 57335 84168671 A  ERRORERROR'
2015-11-09 23:22:48 [DEBUG   ] Got command RPT from MCS: ref #531551
2015-11-09 23:22:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531551  84 57335 84168671 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:23:02 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 23:23:22 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 23:23:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:23:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:23:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:23:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:23:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:23:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:23:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:23:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:23:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:23:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:23:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 23:23:41 [DEBUG   ] Finished updating board flags in 1.539 seconds
2015-11-09 23:23:42 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:24:02 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-09 23:24:22 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 23:24:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:24:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:24:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:24:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:24:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:24:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:24:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:24:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:24:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:24:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:24:41 [DEBUG   ] Finished updating board flags in 1.602 seconds
2015-11-09 23:24:42 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-09 23:25:02 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 23:25:17 [DEBUG   ] Got command RPT from MCS: ref #531571
2015-11-09 23:25:17 [DEBUG   ] summary = ERROR
2015-11-09 23:25:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531571  13 57335 84317935 A  ERRORERROR'
2015-11-09 23:25:18 [DEBUG   ] Got command RPT from MCS: ref #531572
2015-11-09 23:25:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:25:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531572  84 57335 84318139 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:25:18 [DEBUG   ] Got command RPT from MCS: ref #531573
2015-11-09 23:25:18 [DEBUG   ] summary = ERROR
2015-11-09 23:25:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531573  13 57335 84318343 A  ERRORERROR'
2015-11-09 23:25:18 [DEBUG   ] Got command RPT from MCS: ref #531574
2015-11-09 23:25:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:25:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531574  84 57335 84318547 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:25:18 [DEBUG   ] Got command RPT from MCS: ref #531575
2015-11-09 23:25:18 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 23:25:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531575  12 57335 84318739 A  ERRORBŒÌÍ'
2015-11-09 23:25:23 [DEBUG   ] Finished updating ant stats in 1.803 seconds
2015-11-09 23:25:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:25:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:25:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:25:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:25:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:25:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:25:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:25:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:25:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:25:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:25:41 [DEBUG   ] Finished updating board flags in 1.402 seconds
2015-11-09 23:25:42 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 23:26:02 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 23:26:22 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 23:26:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:26:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:26:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:26:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:26:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:26:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:26:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:26:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:26:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:26:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:26:41 [DEBUG   ] Finished updating board flags in 1.436 seconds
2015-11-09 23:26:42 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-09 23:27:02 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 23:27:23 [DEBUG   ] Finished updating ant stats in 1.601 seconds
2015-11-09 23:27:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:27:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:27:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:27:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:27:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:27:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:27:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:27:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:27:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:27:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:27:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 23:27:41 [DEBUG   ] Finished updating board flags in 1.415 seconds
2015-11-09 23:27:42 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 23:27:48 [DEBUG   ] Got command RPT from MCS: ref #531584
2015-11-09 23:27:48 [DEBUG   ] summary = ERROR
2015-11-09 23:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531584  13 57335 84468675 A  ERRORERROR'
2015-11-09 23:27:48 [DEBUG   ] Got command RPT from MCS: ref #531585
2015-11-09 23:27:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531585  84 57335 84468675 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:28:02 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 23:28:22 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 23:28:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:28:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:28:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:28:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:28:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:28:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:28:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:28:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:28:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:28:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:28:41 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 23:28:41 [DEBUG   ] Finished updating board flags in 1.456 seconds
2015-11-09 23:28:42 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 23:29:02 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 23:29:22 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 23:29:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:29:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:29:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:29:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:29:40 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:29:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:29:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:29:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:29:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:29:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:29:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 23:29:41 [DEBUG   ] Finished updating board flags in 1.534 seconds
2015-11-09 23:29:42 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 23:30:02 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 23:30:18 [DEBUG   ] Got command RPT from MCS: ref #531604
2015-11-09 23:30:18 [DEBUG   ] summary = ERROR
2015-11-09 23:30:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531604  13 57335 84618027 A  ERRORERROR'
2015-11-09 23:30:18 [DEBUG   ] Got command RPT from MCS: ref #531605
2015-11-09 23:30:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:30:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531605  84 57335 84618231 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:30:18 [DEBUG   ] Got command RPT from MCS: ref #531606
2015-11-09 23:30:18 [DEBUG   ] summary = ERROR
2015-11-09 23:30:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531606  13 57335 84618435 A  ERRORERROR'
2015-11-09 23:30:18 [DEBUG   ] Got command RPT from MCS: ref #531607
2015-11-09 23:30:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:30:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531607  84 57335 84618627 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:30:18 [DEBUG   ] Got command RPT from MCS: ref #531608
2015-11-09 23:30:18 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 23:30:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531608  12 57335 84618831 A  ERRORB‹ÌÍ'
2015-11-09 23:30:22 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-09 23:30:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998431
2015-11-09 23:30:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998431
2015-11-09 23:30:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998431
2015-11-09 23:30:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998431
2015-11-09 23:30:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998431
2015-11-09 23:30:41 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 23:30:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998431
2015-11-09 23:30:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998431
2015-11-09 23:30:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998431
2015-11-09 23:30:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998431
2015-11-09 23:30:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998431
2015-11-09 23:30:42 [DEBUG   ] Finished updating board flags in 2.191 seconds
2015-11-09 23:30:43 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 23:31:03 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-09 23:31:23 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998333
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998333
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998333
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998333
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998333
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-09 23:31:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998333
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998333
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998333
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998333
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998333
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-09 23:31:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-09 23:31:42 [DEBUG   ] Finished updating board flags in 2.006 seconds
2015-11-09 23:31:43 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 23:32:03 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 23:32:23 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998333
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998333
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998333
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998333
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998333
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 23:32:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998333
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998333
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998333
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998333
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998333
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 23:32:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 23:32:42 [DEBUG   ] Finished updating board flags in 2.009 seconds
2015-11-09 23:32:43 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 23:32:48 [DEBUG   ] Got command RPT from MCS: ref #531612
2015-11-09 23:32:48 [DEBUG   ] summary = ERROR
2015-11-09 23:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531612  13 57335 84768779 A  ERRORERROR'
2015-11-09 23:32:48 [DEBUG   ] Got command RPT from MCS: ref #531613
2015-11-09 23:32:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531613  84 57335 84768779 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:33:03 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 23:33:23 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 23:33:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998333
2015-11-09 23:33:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998333
2015-11-09 23:33:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998333
2015-11-09 23:33:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998333
2015-11-09 23:33:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998333
2015-11-09 23:33:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998333
2015-11-09 23:33:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998333
2015-11-09 23:33:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998333
2015-11-09 23:33:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998333
2015-11-09 23:33:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998333
2015-11-09 23:33:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-09 23:33:42 [DEBUG   ] Finished updating board flags in 2.033 seconds
2015-11-09 23:33:43 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 23:34:03 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 23:34:23 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 23:34:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998333
2015-11-09 23:34:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998333
2015-11-09 23:34:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998333
2015-11-09 23:34:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998333
2015-11-09 23:34:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998333
2015-11-09 23:34:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998333
2015-11-09 23:34:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998333
2015-11-09 23:34:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998333
2015-11-09 23:34:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998333
2015-11-09 23:34:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998333
2015-11-09 23:34:42 [DEBUG   ] Finished updating board flags in 1.933 seconds
2015-11-09 23:34:43 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 23:35:03 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 23:35:18 [DEBUG   ] Got command RPT from MCS: ref #531630
2015-11-09 23:35:18 [DEBUG   ] summary = ERROR
2015-11-09 23:35:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531630  13 57335 84918127 A  ERRORERROR'
2015-11-09 23:35:18 [DEBUG   ] Got command RPT from MCS: ref #531631
2015-11-09 23:35:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:35:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531631  84 57335 84918331 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:35:18 [DEBUG   ] Got command RPT from MCS: ref #531632
2015-11-09 23:35:18 [DEBUG   ] summary = ERROR
2015-11-09 23:35:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531632  13 57335 84918535 A  ERRORERROR'
2015-11-09 23:35:18 [DEBUG   ] Got command RPT from MCS: ref #531633
2015-11-09 23:35:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:35:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531633  84 57335 84918727 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:35:18 [DEBUG   ] Got command RPT from MCS: ref #531634
2015-11-09 23:35:18 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 23:35:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531634  12 57335 84918931 A  ERRORB‹ÌÍ'
2015-11-09 23:35:23 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 23:35:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998333
2015-11-09 23:35:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998333
2015-11-09 23:35:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998333
2015-11-09 23:35:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998333
2015-11-09 23:35:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998333
2015-11-09 23:35:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998333
2015-11-09 23:35:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998333
2015-11-09 23:35:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998333
2015-11-09 23:35:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998333
2015-11-09 23:35:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998333
2015-11-09 23:35:42 [DEBUG   ] Finished updating board flags in 1.809 seconds
2015-11-09 23:35:43 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 23:36:03 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 23:36:23 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 23:36:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998333
2015-11-09 23:36:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998333
2015-11-09 23:36:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998333
2015-11-09 23:36:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998333
2015-11-09 23:36:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998333
2015-11-09 23:36:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998333
2015-11-09 23:36:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998333
2015-11-09 23:36:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998333
2015-11-09 23:36:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998333
2015-11-09 23:36:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998333
2015-11-09 23:36:42 [DEBUG   ] Finished updating board flags in 1.783 seconds
2015-11-09 23:36:43 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 23:37:03 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 23:37:23 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 23:37:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998333
2015-11-09 23:37:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998333
2015-11-09 23:37:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998333
2015-11-09 23:37:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998333
2015-11-09 23:37:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998333
2015-11-09 23:37:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998333
2015-11-09 23:37:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998333
2015-11-09 23:37:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998333
2015-11-09 23:37:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998333
2015-11-09 23:37:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998333
2015-11-09 23:37:42 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 23:37:42 [DEBUG   ] Finished updating board flags in 1.618 seconds
2015-11-09 23:37:43 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 23:37:48 [DEBUG   ] Got command RPT from MCS: ref #531643
2015-11-09 23:37:48 [DEBUG   ] summary = ERROR
2015-11-09 23:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531643  13 57335 85068827 A  ERRORERROR'
2015-11-09 23:37:48 [DEBUG   ] Got command RPT from MCS: ref #531644
2015-11-09 23:37:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531644  84 57335 85068827 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:38:03 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 23:38:23 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-09 23:38:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998333
2015-11-09 23:38:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998333
2015-11-09 23:38:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998333
2015-11-09 23:38:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998333
2015-11-09 23:38:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998333
2015-11-09 23:38:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998333
2015-11-09 23:38:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998333
2015-11-09 23:38:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998333
2015-11-09 23:38:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998333
2015-11-09 23:38:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998333
2015-11-09 23:38:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 23:38:42 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 23:38:42 [DEBUG   ] Finished updating board flags in 1.601 seconds
2015-11-09 23:38:43 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 23:39:03 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 23:39:23 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-09 23:39:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998333
2015-11-09 23:39:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998333
2015-11-09 23:39:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998333
2015-11-09 23:39:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998333
2015-11-09 23:39:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998333
2015-11-09 23:39:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998333
2015-11-09 23:39:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998333
2015-11-09 23:39:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998333
2015-11-09 23:39:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998333
2015-11-09 23:39:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998333
2015-11-09 23:39:42 [DEBUG   ] Finished updating board flags in 1.607 seconds
2015-11-09 23:39:43 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 23:40:03 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 23:40:18 [DEBUG   ] Got command RPT from MCS: ref #531663
2015-11-09 23:40:18 [DEBUG   ] summary = ERROR
2015-11-09 23:40:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531663  13 57335 85218211 A  ERRORERROR'
2015-11-09 23:40:18 [DEBUG   ] Got command RPT from MCS: ref #531664
2015-11-09 23:40:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:40:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531664  84 57335 85218415 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:40:18 [DEBUG   ] Got command RPT from MCS: ref #531665
2015-11-09 23:40:18 [DEBUG   ] summary = ERROR
2015-11-09 23:40:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531665  13 57335 85218619 A  ERRORERROR'
2015-11-09 23:40:18 [DEBUG   ] Got command RPT from MCS: ref #531666
2015-11-09 23:40:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:40:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531666  84 57335 85218823 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:40:19 [DEBUG   ] Got command RPT from MCS: ref #531667
2015-11-09 23:40:19 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 23:40:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531667  12 57335 85219015 A  ERRORB�ÌÍ'
2015-11-09 23:40:23 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-09 23:40:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998333
2015-11-09 23:40:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998333
2015-11-09 23:40:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998333
2015-11-09 23:40:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998333
2015-11-09 23:40:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998333
2015-11-09 23:40:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998333
2015-11-09 23:40:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998333
2015-11-09 23:40:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998333
2015-11-09 23:40:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998333
2015-11-09 23:40:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998333
2015-11-09 23:40:42 [DEBUG   ] Finished updating board flags in 1.556 seconds
2015-11-09 23:40:43 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-09 23:41:03 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 23:41:23 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-09 23:41:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998333
2015-11-09 23:41:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998333
2015-11-09 23:41:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998333
2015-11-09 23:41:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998333
2015-11-09 23:41:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998333
2015-11-09 23:41:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998333
2015-11-09 23:41:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998333
2015-11-09 23:41:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998333
2015-11-09 23:41:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998333
2015-11-09 23:41:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998333
2015-11-09 23:41:42 [DEBUG   ] Finished updating board flags in 1.441 seconds
2015-11-09 23:41:43 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 23:42:03 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-09 23:42:24 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-09 23:42:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998333
2015-11-09 23:42:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998333
2015-11-09 23:42:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998333
2015-11-09 23:42:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998333
2015-11-09 23:42:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998333
2015-11-09 23:42:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998333
2015-11-09 23:42:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998333
2015-11-09 23:42:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998333
2015-11-09 23:42:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998333
2015-11-09 23:42:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998333
2015-11-09 23:42:42 [DEBUG   ] Finished updating board flags in 1.439 seconds
2015-11-09 23:42:44 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 23:42:48 [DEBUG   ] Got command RPT from MCS: ref #531671
2015-11-09 23:42:48 [DEBUG   ] summary = ERROR
2015-11-09 23:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531671  13 57335 85368931 A  ERRORERROR'
2015-11-09 23:42:48 [DEBUG   ] Got command RPT from MCS: ref #531672
2015-11-09 23:42:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531672  84 57335 85368931 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:43:04 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 23:43:24 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 23:43:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998333
2015-11-09 23:43:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998333
2015-11-09 23:43:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998333
2015-11-09 23:43:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998333
2015-11-09 23:43:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998333
2015-11-09 23:43:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998333
2015-11-09 23:43:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998333
2015-11-09 23:43:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998333
2015-11-09 23:43:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998333
2015-11-09 23:43:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998333
2015-11-09 23:43:42 [DEBUG   ] Finished updating board flags in 1.438 seconds
2015-11-09 23:43:44 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-09 23:44:04 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 23:44:24 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:44:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998333
2015-11-09 23:44:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998333
2015-11-09 23:44:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998333
2015-11-09 23:44:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998333
2015-11-09 23:44:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998333
2015-11-09 23:44:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998333
2015-11-09 23:44:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998333
2015-11-09 23:44:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998333
2015-11-09 23:44:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998333
2015-11-09 23:44:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998333
2015-11-09 23:44:42 [DEBUG   ] Finished updating board flags in 1.439 seconds
2015-11-09 23:44:44 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 23:45:04 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 23:45:18 [DEBUG   ] Got command RPT from MCS: ref #531692
2015-11-09 23:45:18 [DEBUG   ] summary = ERROR
2015-11-09 23:45:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531692  13 57335 85518291 A  ERRORERROR'
2015-11-09 23:45:18 [DEBUG   ] Got command RPT from MCS: ref #531693
2015-11-09 23:45:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:45:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531693  84 57335 85518495 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:45:18 [DEBUG   ] Got command RPT from MCS: ref #531694
2015-11-09 23:45:18 [DEBUG   ] summary = ERROR
2015-11-09 23:45:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531694  13 57335 85518699 A  ERRORERROR'
2015-11-09 23:45:18 [DEBUG   ] Got command RPT from MCS: ref #531695
2015-11-09 23:45:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:45:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531695  84 57335 85518903 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:45:19 [DEBUG   ] Got command RPT from MCS: ref #531696
2015-11-09 23:45:19 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 23:45:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531696  12 57335 85519095 A  ERRORB�ÌÍ'
2015-11-09 23:45:24 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-09 23:45:41 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-09 23:45:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 23:45:42 [DEBUG   ] Finished updating board flags in 1.536 seconds
2015-11-09 23:45:44 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:46:04 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 23:46:24 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-09 23:46:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-09 23:46:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-09 23:46:43 [DEBUG   ] Finished updating board flags in 2.215 seconds
2015-11-09 23:46:44 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 23:47:04 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:47:24 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 23:47:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:47:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:47:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:47:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:47:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:47:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:47:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:47:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:47:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:47:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:47:43 [DEBUG   ] Finished updating board flags in 2.144 seconds
2015-11-09 23:47:44 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 23:47:48 [DEBUG   ] Got command RPT from MCS: ref #531705
2015-11-09 23:47:48 [DEBUG   ] summary = ERROR
2015-11-09 23:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531705  13 57335 85668927 A  ERRORERROR'
2015-11-09 23:47:48 [DEBUG   ] Got command RPT from MCS: ref #531706
2015-11-09 23:47:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531706  84 57335 85668927 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:48:04 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 23:48:24 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:48:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:48:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:48:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:48:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:48:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:48:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:48:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:48:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:48:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:48:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:48:43 [DEBUG   ] Finished updating board flags in 2.055 seconds
2015-11-09 23:48:44 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 23:49:04 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-09 23:49:24 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 23:49:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:49:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:49:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:49:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:49:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:49:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:49:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:49:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:49:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:49:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:49:43 [DEBUG   ] Finished updating board flags in 1.952 seconds
2015-11-09 23:49:44 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-09 23:50:04 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-09 23:50:18 [DEBUG   ] Got command RPT from MCS: ref #531722
2015-11-09 23:50:18 [DEBUG   ] summary = ERROR
2015-11-09 23:50:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531722  13 57335 85818387 A  ERRORERROR'
2015-11-09 23:50:18 [DEBUG   ] Got command RPT from MCS: ref #531723
2015-11-09 23:50:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:50:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531723  84 57335 85818595 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:50:18 [DEBUG   ] Got command RPT from MCS: ref #531724
2015-11-09 23:50:18 [DEBUG   ] summary = ERROR
2015-11-09 23:50:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531724  13 57335 85818799 A  ERRORERROR'
2015-11-09 23:50:18 [DEBUG   ] Got command RPT from MCS: ref #531725
2015-11-09 23:50:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:50:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531725  84 57335 85818991 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:50:19 [DEBUG   ] Got command RPT from MCS: ref #531726
2015-11-09 23:50:19 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 23:50:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531726  12 57335 85819195 A  ERRORB�ÌÍ'
2015-11-09 23:50:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 23:50:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:50:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:50:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:50:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:50:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:50:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:50:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:50:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:50:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:50:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:50:43 [DEBUG   ] Finished updating board flags in 1.802 seconds
2015-11-09 23:50:44 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2015-11-09 23:51:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 23:51:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-09 23:51:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:51:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:51:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:51:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:51:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:51:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:51:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:51:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:51:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:51:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:51:43 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-09 23:51:43 [DEBUG   ] Finished updating board flags in 1.781 seconds
2015-11-09 23:51:44 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2015-11-09 23:52:05 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-09 23:52:25 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 23:52:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:52:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:52:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:52:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:52:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:52:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:52:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:52:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:52:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:52:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:52:43 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 23:52:43 [DEBUG   ] Finished updating board flags in 1.578 seconds
2015-11-09 23:52:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 23:52:48 [DEBUG   ] Got command RPT from MCS: ref #531730
2015-11-09 23:52:48 [DEBUG   ] summary = ERROR
2015-11-09 23:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531730  13 57335 85968939 A  ERRORERROR'
2015-11-09 23:52:48 [DEBUG   ] Got command RPT from MCS: ref #531731
2015-11-09 23:52:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531731  84 57335 85968939 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:53:05 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 23:53:25 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-09 23:53:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:53:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:53:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:53:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:53:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:53:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:53:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:53:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:53:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:53:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:53:43 [DEBUG   ] Finished updating board flags in 1.516 seconds
2015-11-09 23:53:45 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 23:54:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-09 23:54:25 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 23:54:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:54:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:54:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:54:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:54:42 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:54:43 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 23:54:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:54:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:54:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:54:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:54:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:54:43 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-09 23:54:43 [DEBUG   ] Finished updating board flags in 1.513 seconds
2015-11-09 23:54:45 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 23:55:05 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-09 23:55:18 [DEBUG   ] Got command RPT from MCS: ref #531751
2015-11-09 23:55:18 [DEBUG   ] summary = ERROR
2015-11-09 23:55:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531751  13 57335 86118479 A  ERRORERROR'
2015-11-09 23:55:18 [DEBUG   ] Got command RPT from MCS: ref #531752
2015-11-09 23:55:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:55:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531752  84 57335 86118683 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:55:18 [DEBUG   ] Got command RPT from MCS: ref #531753
2015-11-09 23:55:18 [DEBUG   ] summary = ERROR
2015-11-09 23:55:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531753  13 57335 86118887 A  ERRORERROR'
2015-11-09 23:55:19 [DEBUG   ] Got command RPT from MCS: ref #531754
2015-11-09 23:55:19 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:55:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531754  84 57335 86119079 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:55:19 [DEBUG   ] Got command RPT from MCS: ref #531755
2015-11-09 23:55:19 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-09 23:55:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531755  12 57335 86119283 A  ERRORB‹ÌÍ'
2015-11-09 23:55:25 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-09 23:55:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:55:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:55:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:55:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:55:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:55:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:55:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:55:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:55:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:55:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:55:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 23:55:44 [DEBUG   ] Finished updating board flags in 2.236 seconds
2015-11-09 23:55:45 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-09 23:56:05 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-09 23:56:25 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-09 23:56:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:56:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:56:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:56:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:56:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:56:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:56:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:56:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:56:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:56:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:56:44 [DEBUG   ] Finished updating board flags in 2.195 seconds
2015-11-09 23:56:45 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-09 23:57:05 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 23:57:25 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 23:57:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:57:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:57:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:57:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:57:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:57:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:57:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:57:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:57:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:57:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:57:44 [DEBUG   ] Finished updating board flags in 2.050 seconds
2015-11-09 23:57:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-09 23:57:48 [DEBUG   ] Got command RPT from MCS: ref #531764
2015-11-09 23:57:48 [DEBUG   ] summary = ERROR
2015-11-09 23:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531764  13 57335 86268939 A  ERRORERROR'
2015-11-09 23:57:48 [DEBUG   ] Got command RPT from MCS: ref #531765
2015-11-09 23:57:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-09 23:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531765  84 57335 86268939 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-09 23:58:05 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 23:58:25 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 23:58:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:58:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:58:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:58:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:58:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:58:43 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 23:58:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:58:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:58:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:58:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:58:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:58:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 23:58:44 [DEBUG   ] Finished updating board flags in 2.020 seconds
2015-11-09 23:58:45 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-09 23:59:05 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-09 23:59:25 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-09 23:59:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-09 23:59:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-09 23:59:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-09 23:59:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-09 23:59:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-09 23:59:43 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-09 23:59:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-09 23:59:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-09 23:59:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-09 23:59:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-09 23:59:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-09 23:59:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-09 23:59:44 [DEBUG   ] Finished updating board flags in 1.900 seconds
2015-11-09 23:59:45 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 00:00:05 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2015-11-10 00:00:18 [DEBUG   ] Got command RPT from MCS: ref #531784
2015-11-10 00:00:18 [DEBUG   ] summary = ERROR
2015-11-10 00:00:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531784  13 57336    18571 A  ERRORERROR'
2015-11-10 00:00:18 [DEBUG   ] Got command RPT from MCS: ref #531785
2015-11-10 00:00:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 00:00:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531785  84 57336    18763 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 00:00:18 [DEBUG   ] Got command RPT from MCS: ref #531786
2015-11-10 00:00:18 [DEBUG   ] summary = ERROR
2015-11-10 00:00:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531786  13 57336    18967 A  ERRORERROR'
2015-11-10 00:00:19 [DEBUG   ] Got command RPT from MCS: ref #531787
2015-11-10 00:00:19 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 00:00:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531787  84 57336    19171 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 00:00:19 [DEBUG   ] Got command RPT from MCS: ref #531788
2015-11-10 00:00:19 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 00:00:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531788  12 57336    19375 A  ERRORBŠÌÍ'
2015-11-10 00:00:25 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 00:00:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-10 00:00:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-10 00:00:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-10 00:00:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-10 00:00:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-10 00:00:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-10 00:00:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-10 00:00:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-10 00:00:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-10 00:00:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-10 00:00:44 [DEBUG   ] Finished updating board flags in 1.797 seconds
2015-11-10 00:00:45 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 00:01:05 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 00:01:25 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 00:01:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-10 00:01:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-10 00:01:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-10 00:01:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-10 00:01:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-10 00:01:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-10 00:01:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-10 00:01:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-10 00:01:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-10 00:01:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-10 00:01:44 [DEBUG   ] Finished updating board flags in 1.590 seconds
2015-11-10 00:01:45 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 00:02:05 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 00:02:25 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 00:02:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-10 00:02:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-10 00:02:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-10 00:02:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-10 00:02:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-10 00:02:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-10 00:02:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-10 00:02:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-10 00:02:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-10 00:02:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-10 00:02:44 [DEBUG   ] Finished updating board flags in 1.447 seconds
2015-11-10 00:02:45 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 00:02:48 [DEBUG   ] Got command RPT from MCS: ref #531792
2015-11-10 00:02:48 [DEBUG   ] summary = ERROR
2015-11-10 00:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531792  13 57336   168935 A  ERRORERROR'
2015-11-10 00:02:48 [DEBUG   ] Got command RPT from MCS: ref #531793
2015-11-10 00:02:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 00:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531793  84 57336   168935 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 00:03:06 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 00:03:26 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 00:03:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-10 00:03:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-10 00:03:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-10 00:03:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-10 00:03:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-10 00:03:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-10 00:03:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-10 00:03:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-10 00:03:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-10 00:03:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-10 00:03:44 [DEBUG   ] Finished updating board flags in 1.442 seconds
2015-11-10 00:03:46 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 00:04:06 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 00:04:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 00:04:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-10 00:04:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-10 00:04:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-10 00:04:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-10 00:04:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-10 00:04:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-10 00:04:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-10 00:04:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-10 00:04:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-10 00:04:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-10 00:04:44 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:04:44 [DEBUG   ] Finished updating board flags in 1.464 seconds
2015-11-10 00:04:46 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2015-11-10 00:05:06 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 00:05:18 [DEBUG   ] Got command RPT from MCS: ref #531810
2015-11-10 00:05:18 [DEBUG   ] summary = ERROR
2015-11-10 00:05:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531810  13 57336   318643 A  ERRORERROR'
2015-11-10 00:05:18 [DEBUG   ] Got command RPT from MCS: ref #531811
2015-11-10 00:05:18 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 00:05:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531811  84 57336   318847 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 00:05:19 [DEBUG   ] Got command RPT from MCS: ref #531812
2015-11-10 00:05:19 [DEBUG   ] summary = ERROR
2015-11-10 00:05:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531812  13 57336   319051 A  ERRORERROR'
2015-11-10 00:05:19 [DEBUG   ] Got command RPT from MCS: ref #531813
2015-11-10 00:05:19 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 00:05:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531813  84 57336   319243 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 00:05:19 [DEBUG   ] Got command RPT from MCS: ref #531814
2015-11-10 00:05:19 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 00:05:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531814  12 57336   319447 A  ERRORBŒÌÍ'
2015-11-10 00:05:26 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 00:05:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-10 00:05:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-10 00:05:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-10 00:05:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-10 00:05:43 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-10 00:05:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-10 00:05:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-10 00:05:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-10 00:05:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-10 00:05:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-10 00:05:44 [DEBUG   ] Finished updating board flags in 1.467 seconds
2015-11-10 00:05:46 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 00:06:06 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 00:06:26 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 00:06:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-10 00:06:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-10 00:06:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-10 00:06:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-10 00:06:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-10 00:06:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-10 00:06:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-10 00:06:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-10 00:06:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-10 00:06:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-10 00:06:45 [DEBUG   ] Finished updating board flags in 2.272 seconds
2015-11-10 00:06:46 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 00:07:06 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 00:07:26 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 00:07:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-10 00:07:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-10 00:07:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-10 00:07:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-10 00:07:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-10 00:07:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-10 00:07:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-10 00:07:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-10 00:07:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-10 00:07:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-10 00:07:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 00:07:45 [DEBUG   ] Finished updating board flags in 2.203 seconds
2015-11-10 00:07:46 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 00:07:48 [DEBUG   ] Got command RPT from MCS: ref #531823
2015-11-10 00:07:48 [DEBUG   ] summary = ERROR
2015-11-10 00:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531823  13 57336   468931 A  ERRORERROR'
2015-11-10 00:07:48 [DEBUG   ] Got command RPT from MCS: ref #531824
2015-11-10 00:07:48 [DEBUG   ] info = T_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 00:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531824  84 57336   468931 A  ERRORT_NOM1 T_NOM3! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 00:08:06 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 00:08:26 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 00:08:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-10 00:08:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-10 00:08:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-10 00:08:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-10 00:08:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-10 00:08:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-10 00:08:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-10 00:08:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-10 00:08:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-10 00:08:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-10 00:08:45 [DEBUG   ] Finished updating board flags in 2.229 seconds
2015-11-10 00:08:46 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 00:09:06 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 00:09:26 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 00:09:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195998137
2015-11-10 00:09:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195998137
2015-11-10 00:09:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195998137
2015-11-10 00:09:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195998137
2015-11-10 00:09:44 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195998137
2015-11-10 00:09:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195998137
2015-11-10 00:09:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195998137
2015-11-10 00:09:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195998137
2015-11-10 00:09:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195998137
2015-11-10 00:09:45 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195998137
2015-11-10 00:09:45 [DEBUG   ] Finished updating board flags in 2.141 seconds
2015-11-10 00:09:46 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 00:10:00 [DEBUG   ] Got command INI from MCS: ref #531843
2015-11-10 00:10:00 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2015-11-10 00:10:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   531843   8 57336   600447 ABOOTING'
2015-11-10 00:10:18 [DEBUG   ] Got command RPT from MCS: ref #531859
2015-11-10 00:10:18 [DEBUG   ] summary = BOOTING
2015-11-10 00:10:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531859  15 57336   618743 ABOOTINGBOOTING'
2015-11-10 00:10:24 [INFO    ] -------------------
2015-11-10 00:10:24 [INFO    ]  LWA board status
2015-11-10 00:10:24 [INFO    ] -------------------
2015-11-10 00:10:24 [INFO    ] DP 1: 2 lwa27
2015-11-10 00:10:24 [INFO    ] DP 1: 3 lwa24
2015-11-10 00:10:24 [INFO    ] DP 1: 4 lwa23
2015-11-10 00:10:24 [INFO    ] DP 1: 5 lwa18
2015-11-10 00:10:24 [INFO    ] DP 1: 6 lwa10
2015-11-10 00:10:24 [INFO    ] DP 1: 7 lwa32
2015-11-10 00:10:24 [INFO    ] DP 1: 8 lwa11
2015-11-10 00:10:24 [INFO    ] DP 1: 9 lwa30
2015-11-10 00:10:24 [INFO    ] DP 1: 10 lwa12
2015-11-10 00:10:24 [INFO    ] DP 1: 11 lwa8
2015-11-10 00:10:24 [INFO    ] DP 1: 12 lwa13
2015-11-10 00:10:24 [INFO    ] DP 1: 13 lwa26
2015-11-10 00:10:24 [INFO    ] DP 1: 14 lwa14
2015-11-10 00:10:24 [INFO    ] DP 1: 16 lwa22
2015-11-10 00:10:24 [INFO    ] DP 1: 17 lwa20
2015-11-10 00:10:24 [INFO    ] DP 1: 18 lwa1
2015-11-10 00:10:24 [INFO    ] DP 1: 19 lwa5
2015-11-10 00:10:24 [INFO    ] DP 1: 20 lwa25
2015-11-10 00:10:24 [INFO    ] DP 1: 21 lwa35
2015-11-10 00:10:24 [INFO    ] DP 1: 22 lwa19
2015-11-10 00:10:24 [INFO    ] DP 1: 23 lwa21
2015-11-10 00:10:24 [INFO    ] DP 1: 24 lwa6
2015-11-10 00:10:24 [INFO    ] DP 1: 25 lwa7
2015-11-10 00:10:24 [INFO    ] DP 1: 26 lwa33
2015-11-10 00:10:24 [INFO    ] DP 1: 27 lwa15
2015-11-10 00:10:24 [INFO    ] DP 1: 28 lwa28
2015-11-10 00:10:24 [INFO    ] DP 2: 1 lwa16
2015-11-10 00:10:24 [INFO    ] DP 2: 15 lwa34
2015-11-10 00:10:24 [INFO    ] Total of 28 boards
2015-11-10 00:10:24 [INFO    ] Total of 260 stands
2015-11-10 00:10:24 [INFO    ] -------------------
2015-11-10 00:10:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16460
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16461
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16462
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16463
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16464
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16465
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16466
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16467
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16468
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16469
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16470
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16471
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16472
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16473
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16474
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16475
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16476
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16477
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16478
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16479
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16480
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16481
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16482
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16483
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16484
2015-11-10 00:10:31 [DEBUG   ] 2015-11-10 00:10:31 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 16485
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16460 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16461 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16462 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16463 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16464 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16465 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16466 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16467 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16468 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16469 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16470 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16471 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16472 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16473 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16474 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16475 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16476 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16477 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16478 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16479 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16480 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16481 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16482 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16483 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16484 exited with status 0
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 16485 exited with status 0
2015-11-10 00:10:44 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] spawn_process: at 2015-11-10 00:10:44 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 16512
2015-11-10 00:10:44 [DEBUG   ] 2015-11-10 00:10:44 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 16513
2015-11-10 00:10:49 [DEBUG   ] 2015-11-10 00:10:49 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 16512 exited with status 0
2015-11-10 00:10:49 [DEBUG   ] 2015-11-10 00:10:49 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 16513 exited with status 0
2015-11-10 00:10:49 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2015-11-10 00:10:49 [DEBUG   ] spawn_process: at 2015-11-10 00:10:49 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2015-11-10 00:10:49 [DEBUG   ] spawn_process: at 2015-11-10 00:10:49 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2015-11-10 00:10:49 [INFO    ] Setting Beam 1 xbars
2015-11-10 00:10:49 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2015-11-10 00:10:49 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2015-11-10 00:10:49 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2015-11-10 00:10:50 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2015-11-10 00:10:50 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2015-11-10 00:10:50 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2015-11-10 00:10:50 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2015-11-10 00:10:50 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2015-11-10 00:10:50 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2015-11-10 00:10:50 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2015-11-10 00:10:50 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2015-11-10 00:10:50 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2015-11-10 00:10:51 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2015-11-10 00:10:51 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2015-11-10 00:10:51 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2015-11-10 00:10:51 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2015-11-10 00:10:51 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2015-11-10 00:10:51 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2015-11-10 00:10:51 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2015-11-10 00:10:51 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2015-11-10 00:10:51 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2015-11-10 00:10:52 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2015-11-10 00:10:52 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2015-11-10 00:10:52 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2015-11-10 00:10:52 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2015-11-10 00:10:52 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2015-11-10 00:10:52 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2015-11-10 00:10:52 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2015-11-10 00:10:52 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2015-11-10 00:10:52 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2015-11-10 00:10:53 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2015-11-10 00:10:53 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2015-11-10 00:10:53 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2015-11-10 00:10:53 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2015-11-10 00:10:53 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2015-11-10 00:10:53 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2015-11-10 00:10:53 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2015-11-10 00:10:53 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2015-11-10 00:10:53 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2015-11-10 00:10:54 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2015-11-10 00:10:54 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2015-11-10 00:10:54 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2015-11-10 00:10:54 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2015-11-10 00:10:54 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2015-11-10 00:10:54 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2015-11-10 00:10:54 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2015-11-10 00:10:54 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2015-11-10 00:10:54 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2015-11-10 00:10:55 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2015-11-10 00:10:55 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2015-11-10 00:10:55 [INFO    ] Setting Beam 2 xbars
2015-11-10 00:10:55 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2015-11-10 00:10:55 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2015-11-10 00:10:55 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2015-11-10 00:10:55 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2015-11-10 00:10:55 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2015-11-10 00:10:55 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2015-11-10 00:10:55 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2015-11-10 00:10:56 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2015-11-10 00:10:56 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2015-11-10 00:10:56 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2015-11-10 00:10:56 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2015-11-10 00:10:56 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2015-11-10 00:10:56 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2015-11-10 00:10:56 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2015-11-10 00:10:56 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2015-11-10 00:10:56 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2015-11-10 00:10:56 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2015-11-10 00:10:57 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2015-11-10 00:10:57 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2015-11-10 00:10:57 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2015-11-10 00:10:57 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2015-11-10 00:10:57 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2015-11-10 00:10:57 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2015-11-10 00:10:57 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2015-11-10 00:10:57 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2015-11-10 00:10:57 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2015-11-10 00:10:58 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2015-11-10 00:10:58 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2015-11-10 00:10:58 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2015-11-10 00:10:58 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2015-11-10 00:10:58 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2015-11-10 00:10:58 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2015-11-10 00:10:58 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2015-11-10 00:10:58 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2015-11-10 00:10:58 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2015-11-10 00:10:59 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2015-11-10 00:10:59 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2015-11-10 00:10:59 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2015-11-10 00:10:59 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2015-11-10 00:10:59 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2015-11-10 00:10:59 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2015-11-10 00:10:59 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2015-11-10 00:10:59 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2015-11-10 00:10:59 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2015-11-10 00:11:00 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2015-11-10 00:11:00 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2015-11-10 00:11:00 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2015-11-10 00:11:00 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2015-11-10 00:11:00 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2015-11-10 00:11:00 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2015-11-10 00:11:00 [INFO    ] Setting Beam 3 xbars
2015-11-10 00:11:00 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2015-11-10 00:11:00 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2015-11-10 00:11:00 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2015-11-10 00:11:01 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2015-11-10 00:11:01 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2015-11-10 00:11:01 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2015-11-10 00:11:01 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2015-11-10 00:11:01 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2015-11-10 00:11:01 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2015-11-10 00:11:01 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2015-11-10 00:11:01 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2015-11-10 00:11:01 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2015-11-10 00:11:01 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2015-11-10 00:11:02 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2015-11-10 00:11:02 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2015-11-10 00:11:02 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2015-11-10 00:11:02 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2015-11-10 00:11:02 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2015-11-10 00:11:02 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2015-11-10 00:11:02 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2015-11-10 00:11:02 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2015-11-10 00:11:02 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2015-11-10 00:11:03 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2015-11-10 00:11:03 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2015-11-10 00:11:03 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2015-11-10 00:11:03 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2015-11-10 00:11:03 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2015-11-10 00:11:03 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2015-11-10 00:11:03 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2015-11-10 00:11:03 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2015-11-10 00:11:03 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2015-11-10 00:11:04 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2015-11-10 00:11:04 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2015-11-10 00:11:04 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2015-11-10 00:11:04 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2015-11-10 00:11:04 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2015-11-10 00:11:04 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2015-11-10 00:11:04 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2015-11-10 00:11:04 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2015-11-10 00:11:04 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2015-11-10 00:11:05 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2015-11-10 00:11:05 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2015-11-10 00:11:05 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2015-11-10 00:11:05 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2015-11-10 00:11:05 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2015-11-10 00:11:05 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2015-11-10 00:11:05 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2015-11-10 00:11:05 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2015-11-10 00:11:05 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2015-11-10 00:11:06 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2015-11-10 00:11:06 [INFO    ] Setting Beam 4 xbars
2015-11-10 00:11:06 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2015-11-10 00:11:06 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2015-11-10 00:11:06 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2015-11-10 00:11:06 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2015-11-10 00:11:06 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2015-11-10 00:11:06 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2015-11-10 00:11:06 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2015-11-10 00:11:06 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2015-11-10 00:11:07 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2015-11-10 00:11:07 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2015-11-10 00:11:07 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2015-11-10 00:11:07 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2015-11-10 00:11:07 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2015-11-10 00:11:07 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2015-11-10 00:11:07 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2015-11-10 00:11:07 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2015-11-10 00:11:07 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2015-11-10 00:11:08 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2015-11-10 00:11:08 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2015-11-10 00:11:08 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2015-11-10 00:11:08 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2015-11-10 00:11:08 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2015-11-10 00:11:08 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2015-11-10 00:11:08 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2015-11-10 00:11:08 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2015-11-10 00:11:08 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2015-11-10 00:11:08 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2015-11-10 00:11:09 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2015-11-10 00:11:09 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2015-11-10 00:11:09 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2015-11-10 00:11:09 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2015-11-10 00:11:09 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2015-11-10 00:11:09 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2015-11-10 00:11:09 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2015-11-10 00:11:09 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2015-11-10 00:11:09 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2015-11-10 00:11:10 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2015-11-10 00:11:10 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2015-11-10 00:11:10 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2015-11-10 00:11:10 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2015-11-10 00:11:10 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2015-11-10 00:11:10 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2015-11-10 00:11:10 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2015-11-10 00:11:10 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2015-11-10 00:11:10 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2015-11-10 00:11:11 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2015-11-10 00:11:11 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2015-11-10 00:11:11 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2015-11-10 00:11:11 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2015-11-10 00:11:11 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2015-11-10 00:11:11 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2015-11-10 00:11:11 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2015-11-10 00:11:11 [INFO    ] All xbars set
2015-11-10 00:11:11 [INFO    ] Setting adder chain routing for all beams
2015-11-10 00:11:16 [INFO    ] Setting beam ids
2015-11-10 00:11:26 [INFO    ] Starting beam delay calibration
2015-11-10 00:11:38 [INFO    ] Beam 1 latency is 5722
2015-11-10 00:11:38 [INFO    ] Beam 2 latency is 5762
2015-11-10 00:11:38 [INFO    ] Beam 3 latency is 5753
2015-11-10 00:11:38 [INFO    ] Beam 4 latency is 5730
2015-11-10 00:11:38 [INFO    ] Stop calibration
2015-11-10 00:11:46 [INFO    ] Beam delay calibration successful
2015-11-10 00:11:46 [INFO    ] Calibration passed
2015-11-10 00:11:46 [DEBUG   ] spawn_process: at 2015-11-10 00:11:46 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2015-11-10 00:11:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2015-11-10 00:11:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2015-11-10 00:11:46 [DEBUG   ] spawn_process: at 2015-11-10 00:11:46 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2015-11-10 00:11:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2015-11-10 00:11:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2015-11-10 00:11:46 [DEBUG   ] spawn_process: at 2015-11-10 00:11:46 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2015-11-10 00:11:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2015-11-10 00:11:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2015-11-10 00:11:47 [DEBUG   ] spawn_process: at 2015-11-10 00:11:47 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2015-11-10 00:11:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2015-11-10 00:11:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2015-11-10 00:11:47 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2015-11-10 00:11:47 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 276 ms (1447114307.278 s) with PID 18074 and time 1447114308
2015-11-10 00:11:47 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2015-11-10 00:11:47 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 366 ms (1447114307.369 s) with PID 18077 and time 1447114308
2015-11-10 00:11:48 [DEBUG   ] Finished updating ant stats in 0.879 seconds
2015-11-10 00:11:48 [INFO    ] Started antennas statistics background thread
2015-11-10 00:11:48 [INFO    ] Started board flags background thread
2015-11-10 00:11:48 [INFO    ] Started BAM background thread
2015-11-10 00:11:48 [INFO    ] Finished the INI process (ref. #531843) in 108.046 s
2015-11-10 00:11:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:11:49 [DEBUG   ] Finished updating board flags in 1.066 seconds
2015-11-10 00:12:08 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 00:12:28 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 00:12:30 [DEBUG   ] Got command RPT from MCS: ref #531886
2015-11-10 00:12:30 [DEBUG   ] summary = NORMAL
2015-11-10 00:12:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531886  14 57336   750443 A NORMALNORMAL'
2015-11-10 00:12:30 [DEBUG   ] Got command RPT from MCS: ref #531887
2015-11-10 00:12:30 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 00:12:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531887  48 57336   750443 A NORMALSystem calibrated and operating normally'
2015-11-10 00:12:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 00:12:48 [DEBUG   ] Got command RPT from MCS: ref #531888
2015-11-10 00:12:48 [DEBUG   ] summary = NORMAL
2015-11-10 00:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531888  14 57336   768939 A NORMALNORMAL'
2015-11-10 00:12:48 [DEBUG   ] Got command RPT from MCS: ref #531889
2015-11-10 00:12:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 00:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531889  48 57336   768939 A NORMALSystem calibrated and operating normally'
2015-11-10 00:12:49 [DEBUG   ] Finished updating board flags in 1.012 seconds
2015-11-10 00:13:08 [DEBUG   ] Finished updating ant stats in 0.592 seconds
2015-11-10 00:13:28 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 00:13:48 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 00:13:49 [DEBUG   ] Finished updating board flags in 0.988 seconds
2015-11-10 00:14:08 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 00:14:29 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 00:14:49 [DEBUG   ] Finished updating ant stats in 1.029 seconds
2015-11-10 00:14:50 [DEBUG   ] Finished updating board flags in 1.816 seconds
2015-11-10 00:15:09 [DEBUG   ] Finished updating ant stats in 0.972 seconds
2015-11-10 00:15:18 [DEBUG   ] Got command RPT from MCS: ref #531909
2015-11-10 00:15:18 [DEBUG   ] summary = NORMAL
2015-11-10 00:15:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531909  14 57336   918831 A NORMALNORMAL'
2015-11-10 00:15:29 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 00:15:49 [DEBUG   ] Finished updating ant stats in 0.824 seconds
2015-11-10 00:15:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 00:15:50 [DEBUG   ] Finished updating board flags in 1.786 seconds
2015-11-10 00:16:09 [DEBUG   ] Finished updating ant stats in 0.772 seconds
2015-11-10 00:16:29 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 00:16:49 [DEBUG   ] Finished updating ant stats in 0.629 seconds
2015-11-10 00:16:50 [DEBUG   ] Finished updating board flags in 1.616 seconds
2015-11-10 00:17:09 [DEBUG   ] Finished updating ant stats in 0.572 seconds
2015-11-10 00:17:29 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 00:17:48 [DEBUG   ] Got command RPT from MCS: ref #531918
2015-11-10 00:17:48 [DEBUG   ] summary = NORMAL
2015-11-10 00:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531918  14 57336  1068943 A NORMALNORMAL'
2015-11-10 00:17:48 [DEBUG   ] Got command RPT from MCS: ref #531919
2015-11-10 00:17:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 00:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531919  48 57336  1068943 A NORMALSystem calibrated and operating normally'
2015-11-10 00:17:50 [DEBUG   ] Finished updating ant stats in 1.226 seconds
2015-11-10 00:17:50 [DEBUG   ] Finished updating board flags in 1.675 seconds
2015-11-10 00:18:10 [DEBUG   ] Finished updating ant stats in 1.176 seconds
2015-11-10 00:18:30 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 00:18:50 [DEBUG   ] Finished updating ant stats in 1.026 seconds
2015-11-10 00:18:50 [DEBUG   ] Finished updating board flags in 1.579 seconds
2015-11-10 00:19:10 [DEBUG   ] Finished updating ant stats in 0.976 seconds
2015-11-10 00:19:30 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 00:19:50 [DEBUG   ] Finished updating ant stats in 0.826 seconds
2015-11-10 00:19:50 [DEBUG   ] Finished updating board flags in 1.416 seconds
2015-11-10 00:20:10 [DEBUG   ] Finished updating ant stats in 0.776 seconds
2015-11-10 00:20:18 [DEBUG   ] Got command RPT from MCS: ref #531935
2015-11-10 00:20:18 [DEBUG   ] summary = NORMAL
2015-11-10 00:20:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531935  14 57336  1218911 A NORMALNORMAL'
2015-11-10 00:20:30 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 00:20:50 [DEBUG   ] Finished updating ant stats in 0.626 seconds
2015-11-10 00:20:50 [DEBUG   ] Finished updating board flags in 1.390 seconds
2015-11-10 00:21:10 [DEBUG   ] Finished updating ant stats in 0.576 seconds
2015-11-10 00:21:30 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 00:21:50 [DEBUG   ] Finished updating ant stats in 0.426 seconds
2015-11-10 00:21:50 [DEBUG   ] Finished updating board flags in 1.186 seconds
2015-11-10 00:22:10 [DEBUG   ] Finished updating ant stats in 0.376 seconds
2015-11-10 00:22:31 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 00:22:48 [DEBUG   ] Got command RPT from MCS: ref #531939
2015-11-10 00:22:48 [DEBUG   ] summary = NORMAL
2015-11-10 00:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531939  14 57336  1368943 A NORMALNORMAL'
2015-11-10 00:22:48 [DEBUG   ] Got command RPT from MCS: ref #531940
2015-11-10 00:22:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 00:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531940  48 57336  1368943 A NORMALSystem calibrated and operating normally'
2015-11-10 00:22:51 [DEBUG   ] Finished updating ant stats in 1.226 seconds
2015-11-10 00:22:51 [DEBUG   ] Finished updating board flags in 2.022 seconds
2015-11-10 00:22:59 [DEBUG   ] Got command DRX from MCS: ref #531963
2015-11-10 00:22:59 [INFO    ] sending drx command to lwa16
2015-11-10 00:22:59 [DEBUG   ] run_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 32000000.0 7 9 26']
2015-11-10 00:22:59 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 00:22:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   531963   8 57336  1379011 A NORMAL'
2015-11-10 00:22:59 [DEBUG   ] Got command DRX from MCS: ref #531964
2015-11-10 00:22:59 [INFO    ] sending drx command to lwa16
2015-11-10 00:22:59 [DEBUG   ] run_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 32000000.0 7 9 26']
2015-11-10 00:22:59 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 00:22:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   531964   8 57336  1379015 A NORMAL'
2015-11-10 00:22:59 [DEBUG   ] Got command DRX from MCS: ref #531962
2015-11-10 00:22:59 [INFO    ] sending drx command to lwa34
2015-11-10 00:22:59 [DEBUG   ] run_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa34', '/lwa/software/drx 1 1 32000000.0 7 9 26']
2015-11-10 00:22:59 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 00:22:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   531962   8 57336  1379019 A NORMAL'
2015-11-10 00:22:59 [DEBUG   ] Got command DRX from MCS: ref #531966
2015-11-10 00:22:59 [INFO    ] sending drx command to lwa16
2015-11-10 00:22:59 [DEBUG   ] run_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 46000000.0 7 9 26']
2015-11-10 00:22:59 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 00:22:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   531966   8 57336  1379023 A NORMAL'
2015-11-10 00:22:59 [DEBUG   ] Got command DRX from MCS: ref #531967
2015-11-10 00:22:59 [INFO    ] sending drx command to lwa16
2015-11-10 00:22:59 [DEBUG   ] run_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 46000000.0 7 9 26']
2015-11-10 00:22:59 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 00:22:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   531967   8 57336  1379027 A NORMAL'
2015-11-10 00:22:59 [DEBUG   ] Got command DRX from MCS: ref #531965
2015-11-10 00:22:59 [INFO    ] sending drx command to lwa34
2015-11-10 00:22:59 [DEBUG   ] run_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa34', '/lwa/software/drx 1 2 46000000.0 7 9 26']
2015-11-10 00:22:59 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 00:22:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   531965   8 57336  1379031 A NORMAL'
2015-11-10 00:22:59 [INFO    ] Finished starting DRX
2015-11-10 00:22:59 [INFO    ] Finished starting DRX
2015-11-10 00:22:59 [INFO    ] Finished starting DRX
2015-11-10 00:22:59 [INFO    ] Finished starting DRX
2015-11-10 00:22:59 [INFO    ] Finished starting DRX
2015-11-10 00:22:59 [INFO    ] Finished starting DRX
2015-11-10 00:22:59 [DEBUG   ] Got command BAM from MCS: ref #531969
2015-11-10 00:22:59 [INFO    ] Finished processing BAM request
2015-11-10 00:22:59 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 00:22:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   531969   8 57336  1379143 A NORMAL'
2015-11-10 00:22:59 [DEBUG   ] Got command BAM from MCS: ref #531970
2015-11-10 00:22:59 [INFO    ] Finished processing BAM request
2015-11-10 00:22:59 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 00:22:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   531970   8 57336  1379231 A NORMAL'
2015-11-10 00:22:59 [DEBUG   ] Got command BAM from MCS: ref #531968
2015-11-10 00:22:59 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 00:22:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   531968   8 57336  1379319 A NORMAL'
2015-11-10 00:22:59 [INFO    ] Finished processing BAM request
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 00:22:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 00:22:59 [DEBUG   ] spawn_process: at 2015-11-10 00:22:59 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 00:23:11 [DEBUG   ] Finished updating ant stats in 1.176 seconds
2015-11-10 00:23:31 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 00:23:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa34, chip 1
2015-11-10 00:23:51 [DEBUG   ] Finished updating ant stats in 1.026 seconds
2015-11-10 00:23:51 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:23:51 [DEBUG   ] Finished updating board flags in 2.013 seconds
2015-11-10 00:24:11 [DEBUG   ] Finished updating ant stats in 0.976 seconds
2015-11-10 00:24:31 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 00:24:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 5
2015-11-10 00:24:51 [DEBUG   ] Finished updating ant stats in 0.826 seconds
2015-11-10 00:24:51 [DEBUG   ] Finished updating board flags in 1.979 seconds
2015-11-10 00:25:00 [DEBUG   ] Got command TBN from MCS: ref #531977
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '1', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '2', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] tbn = exited with status True, code 0x00
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '3', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBN   531977   8 57336  1500456 A NORMAL'
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '4', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '5', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '6', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '7', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '8', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '9', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '10', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '11', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '12', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '13', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '14', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '15', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '16', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '17', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '18', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '19', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '20', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '21', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '22', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '23', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '24', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '25', '1447115101']
2015-11-10 00:25:00 [DEBUG   ] spawn_process: at 2015-11-10 00:25:00 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '26', '1447115101']
2015-11-10 00:25:00 [INFO    ] Finished spawning TBN in 0.054 s
2015-11-10 00:25:11 [DEBUG   ] Finished updating ant stats in 0.844 seconds
2015-11-10 00:25:18 [DEBUG   ] Got command RPT from MCS: ref #531979
2015-11-10 00:25:18 [DEBUG   ] summary = NORMAL
2015-11-10 00:25:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531979  14 57336  1518991 A NORMALNORMAL'
2015-11-10 00:25:31 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 00:25:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 00:25:51 [DEBUG   ] Finished updating ant stats in 0.843 seconds
2015-11-10 00:25:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 00:25:51 [DEBUG   ] Finished updating board flags in 2.036 seconds
2015-11-10 00:26:11 [DEBUG   ] Finished updating ant stats in 0.760 seconds
2015-11-10 00:26:31 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 00:26:51 [DEBUG   ] Finished updating ant stats in 0.643 seconds
2015-11-10 00:26:51 [DEBUG   ] Finished updating board flags in 2.003 seconds
2015-11-10 00:27:11 [DEBUG   ] Finished updating ant stats in 0.559 seconds
2015-11-10 00:27:32 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-10 00:27:48 [DEBUG   ] Got command RPT from MCS: ref #531988
2015-11-10 00:27:48 [DEBUG   ] summary = NORMAL
2015-11-10 00:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531988  14 57336  1668939 A NORMALNORMAL'
2015-11-10 00:27:48 [DEBUG   ] Got command RPT from MCS: ref #531989
2015-11-10 00:27:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 00:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   531989  48 57336  1668939 A NORMALSystem calibrated and operating normally'
2015-11-10 00:27:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 5
2015-11-10 00:27:51 [DEBUG   ] Finished updating board flags in 2.022 seconds
2015-11-10 00:27:52 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-10 00:28:12 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 00:28:32 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-10 00:28:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 5
2015-11-10 00:28:51 [DEBUG   ] Finished updating board flags in 1.968 seconds
2015-11-10 00:28:52 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 00:29:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 00:29:32 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 00:29:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 5
2015-11-10 00:29:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 00:29:51 [DEBUG   ] Finished updating board flags in 1.841 seconds
2015-11-10 00:29:52 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 00:30:12 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2015-11-10 00:30:19 [DEBUG   ] Got command RPT from MCS: ref #532008
2015-11-10 00:30:19 [DEBUG   ] summary = NORMAL
2015-11-10 00:30:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532008  14 57336  1819079 A NORMALNORMAL'
2015-11-10 00:30:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 00:30:51 [DEBUG   ] Finished updating board flags in 1.769 seconds
2015-11-10 00:30:52 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2015-11-10 00:31:12 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 00:31:32 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 00:31:51 [DEBUG   ] Finished updating board flags in 1.612 seconds
2015-11-10 00:31:52 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 00:32:12 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 00:32:32 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 00:32:48 [DEBUG   ] Got command RPT from MCS: ref #532012
2015-11-10 00:32:48 [DEBUG   ] summary = NORMAL
2015-11-10 00:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532012  14 57336  1968943 A NORMALNORMAL'
2015-11-10 00:32:48 [DEBUG   ] Got command RPT from MCS: ref #532013
2015-11-10 00:32:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 00:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532013  48 57336  1968943 A NORMALSystem calibrated and operating normally'
2015-11-10 00:32:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:32:51 [DEBUG   ] Finished updating board flags in 1.610 seconds
2015-11-10 00:32:52 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 00:33:12 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 00:33:32 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 00:33:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa34, chip 1
2015-11-10 00:33:51 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:33:51 [DEBUG   ] Finished updating board flags in 1.566 seconds
2015-11-10 00:33:53 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-10 00:34:13 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 00:34:33 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-10 00:34:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 00:34:51 [DEBUG   ] Finished updating board flags in 1.497 seconds
2015-11-10 00:34:53 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 00:35:13 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-10 00:35:19 [DEBUG   ] Got command RPT from MCS: ref #532030
2015-11-10 00:35:19 [DEBUG   ] summary = NORMAL
2015-11-10 00:35:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532030  14 57336  2119175 A NORMALNORMAL'
2015-11-10 00:35:33 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 00:35:51 [DEBUG   ] Finished updating board flags in 1.500 seconds
2015-11-10 00:35:53 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 00:36:13 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 00:36:33 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 00:36:52 [DEBUG   ] Finished updating board flags in 2.248 seconds
2015-11-10 00:36:53 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 00:37:13 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 00:37:33 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 00:37:48 [DEBUG   ] Got command RPT from MCS: ref #532039
2015-11-10 00:37:48 [DEBUG   ] summary = NORMAL
2015-11-10 00:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532039  14 57336  2268939 A NORMALNORMAL'
2015-11-10 00:37:48 [DEBUG   ] Got command RPT from MCS: ref #532040
2015-11-10 00:37:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 00:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532040  48 57336  2268939 A NORMALSystem calibrated and operating normally'
2015-11-10 00:37:52 [DEBUG   ] Finished updating board flags in 2.172 seconds
2015-11-10 00:37:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 00:38:13 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 00:38:34 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-10 00:38:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 00:38:52 [DEBUG   ] Finished updating board flags in 1.974 seconds
2015-11-10 00:38:54 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 00:39:14 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 00:39:34 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 00:39:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 00:39:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 00:39:52 [DEBUG   ] Finished updating board flags in 1.814 seconds
2015-11-10 00:39:54 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 00:40:14 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 00:40:19 [DEBUG   ] Got command RPT from MCS: ref #532059
2015-11-10 00:40:19 [DEBUG   ] summary = NORMAL
2015-11-10 00:40:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532059  14 57336  2419291 A NORMALNORMAL'
2015-11-10 00:40:34 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 00:40:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 00:40:52 [DEBUG   ] Finished updating board flags in 1.797 seconds
2015-11-10 00:40:54 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 00:41:14 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 00:41:34 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 00:41:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 00:41:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:41:52 [DEBUG   ] Finished updating board flags in 1.623 seconds
2015-11-10 00:41:54 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-10 00:42:14 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 00:42:34 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 00:42:48 [DEBUG   ] Got command RPT from MCS: ref #532063
2015-11-10 00:42:48 [DEBUG   ] summary = NORMAL
2015-11-10 00:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532063  14 57336  2568951 A NORMALNORMAL'
2015-11-10 00:42:48 [DEBUG   ] Got command RPT from MCS: ref #532064
2015-11-10 00:42:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 00:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532064  48 57336  2568951 A NORMALSystem calibrated and operating normally'
2015-11-10 00:42:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 00:42:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 00:42:52 [DEBUG   ] Finished updating board flags in 1.553 seconds
2015-11-10 00:42:54 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 00:43:14 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 00:43:34 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 00:43:52 [DEBUG   ] Finished updating board flags in 1.419 seconds
2015-11-10 00:43:54 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2015-11-10 00:44:14 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 00:44:35 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2015-11-10 00:44:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 00:44:52 [DEBUG   ] Finished updating board flags in 1.393 seconds
2015-11-10 00:44:55 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 00:45:15 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 00:45:19 [DEBUG   ] Got command RPT from MCS: ref #532084
2015-11-10 00:45:19 [DEBUG   ] summary = NORMAL
2015-11-10 00:45:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532084  14 57336  2719403 A NORMALNORMAL'
2015-11-10 00:45:35 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 00:45:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2015-11-10 00:45:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 00:45:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 5
2015-11-10 00:45:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:45:53 [DEBUG   ] Finished updating board flags in 2.211 seconds
2015-11-10 00:45:55 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 00:46:15 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 00:46:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 00:46:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:46:53 [DEBUG   ] Finished updating board flags in 2.183 seconds
2015-11-10 00:46:55 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 00:47:15 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 00:47:35 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2015-11-10 00:47:48 [DEBUG   ] Got command RPT from MCS: ref #532093
2015-11-10 00:47:48 [DEBUG   ] summary = NORMAL
2015-11-10 00:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532093  14 57336  2868951 A NORMALNORMAL'
2015-11-10 00:47:48 [DEBUG   ] Got command RPT from MCS: ref #532094
2015-11-10 00:47:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 00:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532094  48 57336  2868951 A NORMALSystem calibrated and operating normally'
2015-11-10 00:47:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 00:47:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 00:47:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 00:47:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:47:53 [DEBUG   ] Finished updating board flags in 2.021 seconds
2015-11-10 00:47:55 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 00:48:15 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 00:48:35 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 00:48:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:48:53 [DEBUG   ] Finished updating board flags in 2.009 seconds
2015-11-10 00:48:55 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2015-11-10 00:49:15 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 00:49:35 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 00:49:53 [DEBUG   ] Finished updating board flags in 2.029 seconds
2015-11-10 00:49:56 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2015-11-10 00:50:16 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 00:50:19 [DEBUG   ] Got command RPT from MCS: ref #532110
2015-11-10 00:50:19 [DEBUG   ] summary = NORMAL
2015-11-10 00:50:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532110  14 57336  3019499 A NORMALNORMAL'
2015-11-10 00:50:36 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 00:50:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 00:50:53 [DEBUG   ] Finished updating board flags in 2.005 seconds
2015-11-10 00:50:56 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 00:51:16 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2015-11-10 00:51:36 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 00:51:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:51:53 [DEBUG   ] Finished updating board flags in 1.952 seconds
2015-11-10 00:51:56 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 00:52:16 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 00:52:36 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 00:52:48 [DEBUG   ] Got command RPT from MCS: ref #532114
2015-11-10 00:52:48 [DEBUG   ] summary = NORMAL
2015-11-10 00:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532114  14 57336  3168951 A NORMALNORMAL'
2015-11-10 00:52:48 [DEBUG   ] Got command RPT from MCS: ref #532115
2015-11-10 00:52:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 00:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532115  48 57336  3168951 A NORMALSystem calibrated and operating normally'
2015-11-10 00:52:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:52:53 [DEBUG   ] Finished updating board flags in 1.806 seconds
2015-11-10 00:52:56 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 00:53:16 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 00:53:36 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 00:53:53 [DEBUG   ] Finished updating board flags in 1.786 seconds
2015-11-10 00:53:56 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2015-11-10 00:54:17 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2015-11-10 00:54:37 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-10 00:54:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 00:54:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:54:53 [DEBUG   ] Finished updating board flags in 1.625 seconds
2015-11-10 00:54:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 00:55:17 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 00:55:19 [DEBUG   ] Got command RPT from MCS: ref #532135
2015-11-10 00:55:19 [DEBUG   ] summary = NORMAL
2015-11-10 00:55:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532135  14 57336  3319607 A NORMALNORMAL'
2015-11-10 00:55:37 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 00:55:53 [DEBUG   ] Finished updating board flags in 1.586 seconds
2015-11-10 00:55:57 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 00:56:17 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 00:56:37 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 00:56:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 00:56:53 [DEBUG   ] Finished updating board flags in 1.415 seconds
2015-11-10 00:56:57 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 00:57:17 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 00:57:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 00:57:48 [DEBUG   ] Got command RPT from MCS: ref #532144
2015-11-10 00:57:48 [DEBUG   ] summary = NORMAL
2015-11-10 00:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532144  14 57336  3468951 A NORMALNORMAL'
2015-11-10 00:57:48 [DEBUG   ] Got command RPT from MCS: ref #532145
2015-11-10 00:57:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 00:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532145  48 57336  3468951 A NORMALSystem calibrated and operating normally'
2015-11-10 00:57:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 00:57:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 00:57:53 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:57:53 [DEBUG   ] Finished updating board flags in 1.410 seconds
2015-11-10 00:57:57 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 00:58:17 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 00:58:37 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 00:58:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 00:58:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 00:58:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:58:53 [DEBUG   ] Finished updating board flags in 1.375 seconds
2015-11-10 00:58:57 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 00:59:17 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 00:59:38 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-10 00:59:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 00:59:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 00:59:53 [DEBUG   ] Finished updating board flags in 1.366 seconds
2015-11-10 00:59:58 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 01:00:18 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 01:00:19 [DEBUG   ] Got command RPT from MCS: ref #532164
2015-11-10 01:00:19 [DEBUG   ] summary = NORMAL
2015-11-10 01:00:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532164  14 57336  3619699 A NORMALNORMAL'
2015-11-10 01:00:38 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 01:00:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 01:00:54 [DEBUG   ] Finished updating board flags in 2.054 seconds
2015-11-10 01:00:58 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 01:01:18 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 01:01:38 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 01:01:54 [DEBUG   ] Finished updating board flags in 1.990 seconds
2015-11-10 01:01:58 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-10 01:02:18 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 01:02:38 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 01:02:48 [DEBUG   ] Got command RPT from MCS: ref #532168
2015-11-10 01:02:48 [DEBUG   ] summary = NORMAL
2015-11-10 01:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532168  14 57336  3768947 A NORMALNORMAL'
2015-11-10 01:02:48 [DEBUG   ] Got command RPT from MCS: ref #532169
2015-11-10 01:02:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 01:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532169  48 57336  3768947 A NORMALSystem calibrated and operating normally'
2015-11-10 01:02:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 01:02:54 [DEBUG   ] Finished updating board flags in 1.813 seconds
2015-11-10 01:02:59 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 01:03:19 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-10 01:03:39 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 01:03:54 [DEBUG   ] Finished updating board flags in 1.803 seconds
2015-11-10 01:03:59 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 01:04:19 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 01:04:39 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 01:04:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:04:54 [DEBUG   ] Finished updating board flags in 1.857 seconds
2015-11-10 01:04:59 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 01:05:19 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 01:05:19 [DEBUG   ] Got command RPT from MCS: ref #532186
2015-11-10 01:05:19 [DEBUG   ] summary = NORMAL
2015-11-10 01:05:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532186  14 57336  3919803 A NORMALNORMAL'
2015-11-10 01:05:39 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 01:05:54 [DEBUG   ] Finished updating board flags in 1.802 seconds
2015-11-10 01:05:59 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 01:06:19 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-10 01:06:39 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 01:06:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 01:06:54 [DEBUG   ] Finished updating board flags in 1.741 seconds
2015-11-10 01:06:59 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 01:07:19 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 01:07:39 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:07:48 [DEBUG   ] Got command RPT from MCS: ref #532195
2015-11-10 01:07:48 [DEBUG   ] summary = NORMAL
2015-11-10 01:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532195  14 57336  4068947 A NORMALNORMAL'
2015-11-10 01:07:48 [DEBUG   ] Got command RPT from MCS: ref #532196
2015-11-10 01:07:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 01:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532196  48 57336  4068947 A NORMALSystem calibrated and operating normally'
2015-11-10 01:07:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 01:07:54 [DEBUG   ] Finished updating board flags in 1.601 seconds
2015-11-10 01:07:59 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2015-11-10 01:08:20 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 01:08:40 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 01:08:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2015-11-10 01:08:54 [DEBUG   ] Finished updating board flags in 1.611 seconds
2015-11-10 01:09:00 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 01:09:20 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 01:09:40 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 01:09:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 01:09:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:09:54 [DEBUG   ] Finished updating board flags in 1.594 seconds
2015-11-10 01:10:00 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 01:10:19 [DEBUG   ] Got command RPT from MCS: ref #532215
2015-11-10 01:10:19 [DEBUG   ] summary = NORMAL
2015-11-10 01:10:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532215  14 57336  4219879 A NORMALNORMAL'
2015-11-10 01:10:20 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 01:10:40 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 01:10:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 01:10:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:10:54 [DEBUG   ] Finished updating board flags in 1.407 seconds
2015-11-10 01:11:00 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 01:11:20 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 01:11:40 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 01:11:54 [DEBUG   ] Finished updating board flags in 1.449 seconds
2015-11-10 01:12:00 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 01:12:20 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 01:12:40 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 01:12:48 [DEBUG   ] Got command RPT from MCS: ref #532219
2015-11-10 01:12:48 [DEBUG   ] summary = NORMAL
2015-11-10 01:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532219  14 57336  4368955 A NORMALNORMAL'
2015-11-10 01:12:48 [DEBUG   ] Got command RPT from MCS: ref #532220
2015-11-10 01:12:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 01:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532220  48 57336  4368955 A NORMALSystem calibrated and operating normally'
2015-11-10 01:12:54 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:12:54 [DEBUG   ] Finished updating board flags in 1.444 seconds
2015-11-10 01:13:00 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2015-11-10 01:13:20 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 01:13:40 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 01:13:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:13:54 [DEBUG   ] Finished updating board flags in 1.441 seconds
2015-11-10 01:14:00 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:14:20 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 01:14:41 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-10 01:14:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 01:14:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 01:14:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 01:14:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:14:54 [DEBUG   ] Finished updating board flags in 1.440 seconds
2015-11-10 01:15:01 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 01:15:19 [DEBUG   ] Got command RPT from MCS: ref #532240
2015-11-10 01:15:19 [DEBUG   ] summary = NORMAL
2015-11-10 01:15:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532240  14 57336  4519979 A NORMALNORMAL'
2015-11-10 01:15:21 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 01:15:41 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 01:15:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:15:54 [DEBUG   ] Finished updating board flags in 1.445 seconds
2015-11-10 01:16:01 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 01:16:21 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 01:16:41 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 01:16:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:16:55 [DEBUG   ] Finished updating board flags in 2.247 seconds
2015-11-10 01:17:01 [DEBUG   ] Finished updating ant stats in 1.074 seconds
2015-11-10 01:17:21 [DEBUG   ] Finished updating ant stats in 0.930 seconds
2015-11-10 01:17:41 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 01:17:48 [DEBUG   ] Got command RPT from MCS: ref #532249
2015-11-10 01:17:48 [DEBUG   ] summary = NORMAL
2015-11-10 01:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532249  14 57336  4668955 A NORMALNORMAL'
2015-11-10 01:17:48 [DEBUG   ] Got command RPT from MCS: ref #532250
2015-11-10 01:17:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 01:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532250  48 57336  4668955 A NORMALSystem calibrated and operating normally'
2015-11-10 01:17:55 [DEBUG   ] Finished updating board flags in 2.134 seconds
2015-11-10 01:18:01 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:18:21 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 01:18:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 01:18:55 [DEBUG   ] Finished updating board flags in 2.010 seconds
2015-11-10 01:19:01 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:19:21 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 01:19:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 01:19:55 [DEBUG   ] Finished updating board flags in 1.995 seconds
2015-11-10 01:20:01 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 01:20:20 [DEBUG   ] Got command RPT from MCS: ref #532266
2015-11-10 01:20:20 [DEBUG   ] summary = NORMAL
2015-11-10 01:20:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532266  14 57336  4820067 A NORMALNORMAL'
2015-11-10 01:20:22 [DEBUG   ] Finished updating ant stats in 1.390 seconds
2015-11-10 01:20:42 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-10 01:20:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2015-11-10 01:20:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 01:20:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 01:20:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:20:55 [DEBUG   ] Finished updating board flags in 1.823 seconds
2015-11-10 01:21:02 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 01:21:22 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 01:21:42 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-10 01:21:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-10 01:21:55 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:21:55 [DEBUG   ] Finished updating board flags in 1.792 seconds
2015-11-10 01:22:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 01:22:22 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 01:22:42 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-10 01:22:48 [DEBUG   ] Got command RPT from MCS: ref #532270
2015-11-10 01:22:48 [DEBUG   ] summary = NORMAL
2015-11-10 01:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532270  14 57336  4968955 A NORMALNORMAL'
2015-11-10 01:22:48 [DEBUG   ] Got command RPT from MCS: ref #532271
2015-11-10 01:22:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 01:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532271  48 57336  4968955 A NORMALSystem calibrated and operating normally'
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-10 01:22:54 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-10 01:22:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:22:55 [DEBUG   ] Finished updating board flags in 1.623 seconds
2015-11-10 01:23:02 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-10 01:23:22 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 01:23:42 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 01:23:55 [DEBUG   ] Finished updating board flags in 1.558 seconds
2015-11-10 01:24:02 [DEBUG   ] Finished updating ant stats in 0.795 seconds
2015-11-10 01:24:22 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 01:24:42 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 01:24:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 01:24:55 [DEBUG   ] Finished updating board flags in 1.411 seconds
2015-11-10 01:25:02 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:25:20 [DEBUG   ] Got command RPT from MCS: ref #532291
2015-11-10 01:25:20 [DEBUG   ] summary = NORMAL
2015-11-10 01:25:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532291  14 57336  5120171 A NORMALNORMAL'
2015-11-10 01:25:22 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 01:25:42 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:25:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa34, chip 1
2015-11-10 01:25:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:25:55 [DEBUG   ] Finished updating board flags in 1.463 seconds
2015-11-10 01:26:02 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:26:22 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:26:42 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 01:26:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:26:55 [DEBUG   ] Finished updating board flags in 1.491 seconds
2015-11-10 01:27:03 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 01:27:23 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-10 01:27:43 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 01:27:48 [DEBUG   ] Got command RPT from MCS: ref #532300
2015-11-10 01:27:48 [DEBUG   ] summary = NORMAL
2015-11-10 01:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532300  14 57336  5268959 A NORMALNORMAL'
2015-11-10 01:27:48 [DEBUG   ] Got command RPT from MCS: ref #532301
2015-11-10 01:27:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 01:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532301  48 57336  5268959 A NORMALSystem calibrated and operating normally'
2015-11-10 01:27:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa34, chip 1
2015-11-10 01:27:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 01:27:55 [DEBUG   ] Finished updating board flags in 1.486 seconds
2015-11-10 01:28:03 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 01:28:23 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 01:28:43 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 01:28:56 [DEBUG   ] Finished updating board flags in 2.254 seconds
2015-11-10 01:29:03 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 01:29:23 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 01:29:43 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 01:29:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 01:29:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:29:56 [DEBUG   ] Finished updating board flags in 2.223 seconds
2015-11-10 01:30:03 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 01:30:20 [DEBUG   ] Got command RPT from MCS: ref #532320
2015-11-10 01:30:20 [DEBUG   ] summary = NORMAL
2015-11-10 01:30:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532320  14 57336  5420283 A NORMALNORMAL'
2015-11-10 01:30:23 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 01:30:43 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 01:30:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 5
2015-11-10 01:30:56 [DEBUG   ] Finished updating board flags in 2.110 seconds
2015-11-10 01:31:03 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-10 01:31:23 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 01:31:43 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 01:31:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:31:56 [DEBUG   ] Finished updating board flags in 1.995 seconds
2015-11-10 01:32:03 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 01:32:23 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 01:32:43 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 01:32:48 [DEBUG   ] Got command RPT from MCS: ref #532324
2015-11-10 01:32:48 [DEBUG   ] summary = NORMAL
2015-11-10 01:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532324  14 57336  5568959 A NORMALNORMAL'
2015-11-10 01:32:48 [DEBUG   ] Got command RPT from MCS: ref #532325
2015-11-10 01:32:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 01:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532325  48 57336  5568959 A NORMALSystem calibrated and operating normally'
2015-11-10 01:32:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 01:32:56 [DEBUG   ] Finished updating board flags in 1.789 seconds
2015-11-10 01:33:03 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 01:33:23 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 01:33:43 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 01:33:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 01:33:56 [DEBUG   ] Finished updating board flags in 1.621 seconds
2015-11-10 01:34:03 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 01:34:23 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 01:34:43 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 01:34:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 01:34:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:34:56 [DEBUG   ] Finished updating board flags in 1.584 seconds
2015-11-10 01:35:03 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 01:35:20 [DEBUG   ] Got command RPT from MCS: ref #532342
2015-11-10 01:35:20 [DEBUG   ] summary = NORMAL
2015-11-10 01:35:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532342  14 57336  5720383 A NORMALNORMAL'
2015-11-10 01:35:23 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 01:35:43 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 01:35:56 [DEBUG   ] Finished updating board flags in 1.411 seconds
2015-11-10 01:36:03 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 01:36:23 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 01:36:43 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 01:36:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 01:36:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:36:56 [DEBUG   ] Finished updating board flags in 1.588 seconds
2015-11-10 01:37:03 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 01:37:23 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 01:37:43 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 01:37:48 [DEBUG   ] Got command RPT from MCS: ref #532351
2015-11-10 01:37:48 [DEBUG   ] summary = NORMAL
2015-11-10 01:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532351  14 57336  5868963 A NORMALNORMAL'
2015-11-10 01:37:48 [DEBUG   ] Got command RPT from MCS: ref #532352
2015-11-10 01:37:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 01:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532352  48 57336  5868963 A NORMALSystem calibrated and operating normally'
2015-11-10 01:37:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2015-11-10 01:37:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:37:57 [DEBUG   ] Finished updating board flags in 1.611 seconds
2015-11-10 01:38:03 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 01:38:23 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:38:43 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:38:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 01:38:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 01:38:57 [DEBUG   ] Finished updating board flags in 2.320 seconds
2015-11-10 01:39:03 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 01:39:23 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:39:43 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 01:39:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2015-11-10 01:39:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa34, chip 1
2015-11-10 01:39:57 [DEBUG   ] Finished updating board flags in 2.082 seconds
2015-11-10 01:40:03 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:40:20 [DEBUG   ] Got command RPT from MCS: ref #532371
2015-11-10 01:40:20 [DEBUG   ] summary = NORMAL
2015-11-10 01:40:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532371  14 57336  6020483 A NORMALNORMAL'
2015-11-10 01:40:23 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 01:40:43 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-10 01:40:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-10 01:40:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-10 01:40:57 [DEBUG   ] Finished updating board flags in 2.041 seconds
2015-11-10 01:41:03 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:41:23 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:41:43 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-10 01:41:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-10 01:41:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:41:57 [DEBUG   ] Finished updating board flags in 1.971 seconds
2015-11-10 01:42:03 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 01:42:23 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 01:42:43 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:42:48 [DEBUG   ] Got command RPT from MCS: ref #532375
2015-11-10 01:42:48 [DEBUG   ] summary = NORMAL
2015-11-10 01:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532375  14 57336  6168967 A NORMALNORMAL'
2015-11-10 01:42:48 [DEBUG   ] Got command RPT from MCS: ref #532376
2015-11-10 01:42:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 01:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532376  48 57336  6168967 A NORMALSystem calibrated and operating normally'
2015-11-10 01:42:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 01:42:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 01:42:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 01:42:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 01:42:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 01:42:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 01:42:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 01:42:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 01:42:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 01:42:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 01:42:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 01:42:57 [DEBUG   ] Finished updating board flags in 1.799 seconds
2015-11-10 01:43:03 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 01:43:23 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:43:43 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:43:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 01:43:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 01:43:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 01:43:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 01:43:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 01:43:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 01:43:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 01:43:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 01:43:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 01:43:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 01:43:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 01:43:57 [DEBUG   ] Finished updating board flags in 1.713 seconds
2015-11-10 01:44:03 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 01:44:23 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:44:43 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 01:44:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 01:44:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 01:44:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 01:44:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 01:44:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 01:44:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 01:44:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 01:44:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 01:44:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 01:44:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 01:44:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 01:44:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 01:44:57 [DEBUG   ] Finished updating board flags in 1.478 seconds
2015-11-10 01:45:03 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:45:20 [DEBUG   ] Got command RPT from MCS: ref #532396
2015-11-10 01:45:20 [DEBUG   ] summary = NORMAL
2015-11-10 01:45:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532396  14 57336  6320571 A NORMALNORMAL'
2015-11-10 01:45:23 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:45:43 [DEBUG   ] Finished updating ant stats in 0.538 seconds
2015-11-10 01:45:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 01:45:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 01:45:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 01:45:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 01:45:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 01:45:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 01:45:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 01:45:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 01:45:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 01:45:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 01:45:57 [DEBUG   ] Finished updating board flags in 1.486 seconds
2015-11-10 01:46:04 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 01:46:24 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 01:46:44 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 01:46:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 01:46:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 01:46:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 01:46:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 01:46:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 01:46:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 01:46:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 01:46:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 01:46:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 01:46:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 01:46:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 01:46:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:46:57 [DEBUG   ] Finished updating board flags in 1.487 seconds
2015-11-10 01:47:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 01:47:24 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 01:47:44 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 01:47:48 [DEBUG   ] Got command RPT from MCS: ref #532405
2015-11-10 01:47:48 [DEBUG   ] summary = NORMAL
2015-11-10 01:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532405  14 57336  6468959 A NORMALNORMAL'
2015-11-10 01:47:48 [DEBUG   ] Got command RPT from MCS: ref #532406
2015-11-10 01:47:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 01:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532406  48 57336  6468959 A NORMALSystem calibrated and operating normally'
2015-11-10 01:47:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 01:47:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 01:47:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 01:47:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 01:47:56 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 01:47:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 01:47:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 01:47:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 01:47:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 01:47:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 01:47:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:47:57 [DEBUG   ] Finished updating board flags in 1.495 seconds
2015-11-10 01:48:04 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 01:48:24 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 01:48:44 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 01:48:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 01:48:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 01:48:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 01:48:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 01:48:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 01:48:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 01:48:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 01:48:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 01:48:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 01:48:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 01:48:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:48:58 [DEBUG   ] Finished updating board flags in 2.158 seconds
2015-11-10 01:49:04 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 01:49:24 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 01:49:44 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 01:49:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 01:49:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 01:49:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 01:49:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 01:49:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 01:49:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 01:49:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 01:49:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 01:49:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 01:49:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 01:49:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:49:58 [DEBUG   ] Finished updating board flags in 2.008 seconds
2015-11-10 01:50:04 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 01:50:20 [DEBUG   ] Got command RPT from MCS: ref #532422
2015-11-10 01:50:20 [DEBUG   ] summary = NORMAL
2015-11-10 01:50:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532422  14 57336  6620671 A NORMALNORMAL'
2015-11-10 01:50:24 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 01:50:44 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 01:50:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 01:50:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 01:50:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 01:50:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 01:50:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 01:50:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 01:50:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 01:50:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 01:50:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 01:50:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 01:50:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 01:50:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:50:58 [DEBUG   ] Finished updating board flags in 2.007 seconds
2015-11-10 01:51:04 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 01:51:24 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 01:51:44 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-10 01:51:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-10 01:51:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:51:58 [DEBUG   ] Finished updating board flags in 2.029 seconds
2015-11-10 01:52:04 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2015-11-10 01:52:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 01:52:44 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-10 01:52:48 [DEBUG   ] Got command RPT from MCS: ref #532426
2015-11-10 01:52:48 [DEBUG   ] summary = NORMAL
2015-11-10 01:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532426  14 57336  6768959 A NORMALNORMAL'
2015-11-10 01:52:48 [DEBUG   ] Got command RPT from MCS: ref #532427
2015-11-10 01:52:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 01:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532427  48 57336  6768959 A NORMALSystem calibrated and operating normally'
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-10 01:52:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-10 01:52:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-10 01:52:58 [DEBUG   ] Finished updating board flags in 1.955 seconds
2015-11-10 01:53:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 01:53:24 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 01:53:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:53:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 01:53:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 01:53:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 01:53:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 01:53:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 01:53:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2015-11-10 01:53:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 01:53:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 01:53:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 01:53:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 01:53:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 01:53:58 [DEBUG   ] Finished updating board flags in 1.816 seconds
2015-11-10 01:54:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:54:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:54:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:54:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 01:54:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 01:54:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 01:54:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 01:54:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 01:54:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 01:54:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 01:54:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 01:54:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 01:54:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 01:54:58 [DEBUG   ] Finished updating board flags in 1.818 seconds
2015-11-10 01:55:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 01:55:20 [DEBUG   ] Got command RPT from MCS: ref #532447
2015-11-10 01:55:20 [DEBUG   ] summary = NORMAL
2015-11-10 01:55:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532447  14 57336  6920755 A NORMALNORMAL'
2015-11-10 01:55:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:55:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 01:55:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 01:55:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 01:55:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 01:55:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 01:55:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 01:55:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 01:55:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 01:55:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 01:55:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 01:55:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 01:55:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:55:58 [DEBUG   ] Finished updating board flags in 1.823 seconds
2015-11-10 01:56:04 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 01:56:25 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 01:56:45 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-10 01:56:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 01:56:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 01:56:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 01:56:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 01:56:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 01:56:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 01:56:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 01:56:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 01:56:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 01:56:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 01:56:58 [DEBUG   ] Finished updating board flags in 1.744 seconds
2015-11-10 01:57:05 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 01:57:25 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 01:57:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 01:57:48 [DEBUG   ] Got command RPT from MCS: ref #532456
2015-11-10 01:57:48 [DEBUG   ] summary = NORMAL
2015-11-10 01:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532456  14 57336  7068967 A NORMALNORMAL'
2015-11-10 01:57:48 [DEBUG   ] Got command RPT from MCS: ref #532457
2015-11-10 01:57:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 01:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532457  48 57336  7068967 A NORMALSystem calibrated and operating normally'
2015-11-10 01:57:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 01:57:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 01:57:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 01:57:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 01:57:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 01:57:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 01:57:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 01:57:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 01:57:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 01:57:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 01:57:58 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:57:58 [DEBUG   ] Finished updating board flags in 1.613 seconds
2015-11-10 01:58:05 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 01:58:25 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 01:58:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 01:58:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 01:58:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 01:58:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 01:58:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 01:58:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 01:58:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 01:58:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 01:58:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 01:58:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 01:58:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 01:58:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 01:58:58 [DEBUG   ] Finished updating board flags in 1.649 seconds
2015-11-10 01:59:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 01:59:25 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 01:59:45 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 01:59:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 01:59:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 01:59:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 01:59:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 01:59:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 01:59:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 01:59:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 01:59:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 01:59:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 01:59:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 01:59:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 01:59:58 [DEBUG   ] Finished updating board flags in 1.565 seconds
2015-11-10 02:00:05 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 02:00:20 [DEBUG   ] Got command RPT from MCS: ref #532476
2015-11-10 02:00:20 [DEBUG   ] summary = NORMAL
2015-11-10 02:00:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532476  14 57336  7220871 A NORMALNORMAL'
2015-11-10 02:00:25 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 02:00:45 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-10 02:00:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-10 02:00:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-10 02:00:58 [DEBUG   ] Finished updating board flags in 1.518 seconds
2015-11-10 02:01:05 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 02:01:25 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 02:01:45 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-10 02:01:57 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-10 02:01:58 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:01:58 [DEBUG   ] Finished updating board flags in 1.491 seconds
2015-11-10 02:02:05 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 02:02:25 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 02:02:45 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 02:02:48 [DEBUG   ] Got command RPT from MCS: ref #532480
2015-11-10 02:02:48 [DEBUG   ] summary = NORMAL
2015-11-10 02:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532480  14 57336  7368967 A NORMALNORMAL'
2015-11-10 02:02:48 [DEBUG   ] Got command RPT from MCS: ref #532481
2015-11-10 02:02:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 02:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532481  48 57336  7368967 A NORMALSystem calibrated and operating normally'
2015-11-10 02:02:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:02:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:02:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:02:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:02:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:02:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:02:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:02:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:02:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:02:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:02:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 02:02:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:02:59 [DEBUG   ] Finished updating board flags in 2.202 seconds
2015-11-10 02:03:06 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 02:03:26 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 02:03:46 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 02:03:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:03:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:03:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:03:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:03:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:03:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:03:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:03:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:03:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:03:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:03:59 [WARNING ] MonitorFlags: VCCINT 0.976 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:03:59 [DEBUG   ] Finished updating board flags in 2.164 seconds
2015-11-10 02:04:06 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:04:26 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 02:04:46 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:04:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:04:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:04:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:04:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:04:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:04:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 02:04:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:04:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:04:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:04:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:04:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:04:59 [DEBUG   ] Finished updating board flags in 2.067 seconds
2015-11-10 02:05:06 [DEBUG   ] Finished updating ant stats in 1.195 seconds
2015-11-10 02:05:20 [DEBUG   ] Got command RPT from MCS: ref #532498
2015-11-10 02:05:20 [DEBUG   ] summary = NORMAL
2015-11-10 02:05:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532498  14 57336  7520955 A NORMALNORMAL'
2015-11-10 02:05:26 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 02:05:46 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 02:05:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:05:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:05:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:05:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:05:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:05:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:05:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:05:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:05:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:05:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:05:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:05:59 [DEBUG   ] Finished updating board flags in 1.950 seconds
2015-11-10 02:06:06 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 02:06:26 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 02:06:46 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-10 02:06:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:06:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:06:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:06:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:06:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:06:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:06:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:06:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:06:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:06:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:06:59 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:06:59 [DEBUG   ] Finished updating board flags in 1.800 seconds
2015-11-10 02:07:06 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 02:07:26 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 02:07:46 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 02:07:48 [DEBUG   ] Got command RPT from MCS: ref #532507
2015-11-10 02:07:48 [DEBUG   ] summary = NORMAL
2015-11-10 02:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532507  14 57336  7668963 A NORMALNORMAL'
2015-11-10 02:07:48 [DEBUG   ] Got command RPT from MCS: ref #532508
2015-11-10 02:07:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 02:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532508  48 57336  7668963 A NORMALSystem calibrated and operating normally'
2015-11-10 02:07:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:07:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:07:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:07:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:07:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:07:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:07:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:07:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:07:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:07:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:07:59 [DEBUG   ] Finished updating board flags in 1.812 seconds
2015-11-10 02:08:06 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 02:08:26 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 02:08:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 02:08:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:08:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:08:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:08:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:08:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:08:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:08:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:08:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:08:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:08:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:08:59 [DEBUG   ] Finished updating board flags in 1.782 seconds
2015-11-10 02:09:06 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2015-11-10 02:09:27 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2015-11-10 02:09:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:09:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:09:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:09:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:09:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:09:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:09:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:09:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:09:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:09:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:09:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:09:59 [DEBUG   ] Finished updating board flags in 1.613 seconds
2015-11-10 02:10:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:10:21 [DEBUG   ] Got command RPT from MCS: ref #532527
2015-11-10 02:10:21 [DEBUG   ] summary = NORMAL
2015-11-10 02:10:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532527  14 57336  7821063 A NORMALNORMAL'
2015-11-10 02:10:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:10:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:10:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:10:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:10:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:10:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:10:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:10:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:10:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:10:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:10:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:10:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:10:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 02:10:59 [DEBUG   ] Finished updating board flags in 1.596 seconds
2015-11-10 02:11:07 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:11:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:11:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:11:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:11:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:11:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:11:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:11:58 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:11:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:11:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:11:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:11:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:11:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:11:59 [DEBUG   ] Finished updating board flags in 1.400 seconds
2015-11-10 02:12:07 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 02:12:27 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:12:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:12:48 [DEBUG   ] Got command RPT from MCS: ref #532531
2015-11-10 02:12:48 [DEBUG   ] summary = NORMAL
2015-11-10 02:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532531  14 57336  7968967 A NORMALNORMAL'
2015-11-10 02:12:48 [DEBUG   ] Got command RPT from MCS: ref #532532
2015-11-10 02:12:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 02:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532532  48 57336  7968967 A NORMALSystem calibrated and operating normally'
2015-11-10 02:12:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:12:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:12:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:12:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:12:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:13:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:13:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:13:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:13:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:13:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:13:00 [DEBUG   ] Finished updating board flags in 2.211 seconds
2015-11-10 02:13:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:13:27 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 02:13:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:13:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:13:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:13:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:13:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:13:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:14:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:14:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:14:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:14:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:14:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:14:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 02:14:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 02:14:00 [DEBUG   ] Finished updating board flags in 2.262 seconds
2015-11-10 02:14:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:14:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:14:47 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 02:14:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:14:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:14:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:14:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:14:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:15:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:15:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:15:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:15:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:15:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:15:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 02:15:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:15:00 [DEBUG   ] Finished updating board flags in 2.191 seconds
2015-11-10 02:15:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:15:21 [DEBUG   ] Got command RPT from MCS: ref #532552
2015-11-10 02:15:21 [DEBUG   ] summary = NORMAL
2015-11-10 02:15:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532552  14 57336  8121147 A NORMALNORMAL'
2015-11-10 02:15:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:15:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:15:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:15:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:15:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:15:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:15:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:16:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:16:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:16:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:16:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:16:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:16:00 [DEBUG   ] Finished updating board flags in 2.026 seconds
2015-11-10 02:16:07 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:16:27 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 02:16:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:16:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:16:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:16:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:16:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:16:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:17:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:17:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:17:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:17:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:17:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:17:00 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:17:00 [DEBUG   ] Finished updating board flags in 1.917 seconds
2015-11-10 02:17:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:17:27 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 02:17:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:17:48 [DEBUG   ] Got command RPT from MCS: ref #532561
2015-11-10 02:17:48 [DEBUG   ] summary = NORMAL
2015-11-10 02:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532561  14 57336  8268967 A NORMALNORMAL'
2015-11-10 02:17:48 [DEBUG   ] Got command RPT from MCS: ref #532562
2015-11-10 02:17:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 02:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532562  48 57336  8268967 A NORMALSystem calibrated and operating normally'
2015-11-10 02:17:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:17:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:17:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:17:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:17:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:18:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:18:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:18:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:18:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:18:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:18:00 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:18:00 [DEBUG   ] Finished updating board flags in 1.817 seconds
2015-11-10 02:18:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:18:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:18:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:18:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:18:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:18:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:18:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:18:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:19:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:19:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:19:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:19:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:19:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:19:00 [DEBUG   ] Finished updating board flags in 1.857 seconds
2015-11-10 02:19:07 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 02:19:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:19:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:19:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:19:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:19:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:19:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:19:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:19:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2015-11-10 02:20:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:20:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:20:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:20:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:20:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:20:00 [DEBUG   ] Finished updating board flags in 1.722 seconds
2015-11-10 02:20:07 [DEBUG   ] Finished updating ant stats in 1.191 seconds
2015-11-10 02:20:21 [DEBUG   ] Got command RPT from MCS: ref #532578
2015-11-10 02:20:21 [DEBUG   ] summary = NORMAL
2015-11-10 02:20:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532578  14 57336  8421259 A NORMALNORMAL'
2015-11-10 02:20:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 02:20:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 02:20:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:20:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:20:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:20:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:20:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:20:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 02:20:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 02:21:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:21:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:21:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:21:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:21:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:21:00 [DEBUG   ] Finished updating board flags in 1.618 seconds
2015-11-10 02:21:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 02:21:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 02:21:47 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 02:21:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:21:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:21:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:21:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:21:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:21:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 02:21:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 02:22:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:22:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:22:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:22:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:22:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:22:00 [DEBUG   ] Finished updating board flags in 1.592 seconds
2015-11-10 02:22:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 02:22:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 02:22:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 02:22:48 [DEBUG   ] Got command RPT from MCS: ref #532582
2015-11-10 02:22:48 [DEBUG   ] summary = NORMAL
2015-11-10 02:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532582  14 57336  8568971 A NORMALNORMAL'
2015-11-10 02:22:48 [DEBUG   ] Got command RPT from MCS: ref #532583
2015-11-10 02:22:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 02:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532583  48 57336  8568971 A NORMALSystem calibrated and operating normally'
2015-11-10 02:22:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:22:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:22:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:22:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:22:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:23:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:23:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:23:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:23:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:23:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:23:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:23:00 [DEBUG   ] Finished updating board flags in 1.488 seconds
2015-11-10 02:23:07 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 02:23:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 02:23:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 02:23:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:23:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:23:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:23:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:23:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:24:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:24:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:24:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:24:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:24:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:24:00 [DEBUG   ] Finished updating board flags in 1.503 seconds
2015-11-10 02:24:07 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 02:24:27 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 02:24:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 02:24:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:24:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:24:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:24:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:24:59 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:25:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:25:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:25:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:25:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:25:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:25:00 [DEBUG   ] Finished updating board flags in 1.521 seconds
2015-11-10 02:25:07 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 02:25:21 [DEBUG   ] Got command RPT from MCS: ref #532603
2015-11-10 02:25:21 [DEBUG   ] summary = NORMAL
2015-11-10 02:25:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532603  14 57336  8721335 A NORMALNORMAL'
2015-11-10 02:25:28 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-10 02:25:48 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 02:26:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:26:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:26:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:26:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:26:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:26:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 02:26:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:26:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:26:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:26:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:26:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:26:01 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 02:26:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:26:01 [DEBUG   ] Finished updating board flags in 2.137 seconds
2015-11-10 02:26:08 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:26:28 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:26:48 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:27:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:27:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:27:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:27:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:27:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:27:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2015-11-10 02:27:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:27:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:27:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:27:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:27:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:27:01 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:27:01 [DEBUG   ] Finished updating board flags in 2.056 seconds
2015-11-10 02:27:08 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:27:28 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:27:48 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 02:27:48 [DEBUG   ] Got command RPT from MCS: ref #532612
2015-11-10 02:27:48 [DEBUG   ] summary = NORMAL
2015-11-10 02:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532612  14 57336  8868967 A NORMALNORMAL'
2015-11-10 02:27:48 [DEBUG   ] Got command RPT from MCS: ref #532613
2015-11-10 02:27:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 02:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532613  48 57336  8868967 A NORMALSystem calibrated and operating normally'
2015-11-10 02:28:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:28:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:28:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:28:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:28:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:28:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:28:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:28:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:28:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:28:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:28:01 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:28:01 [DEBUG   ] Finished updating board flags in 2.024 seconds
2015-11-10 02:28:08 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:28:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:28:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:29:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:29:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:29:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:29:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:29:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:29:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 02:29:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 02:29:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:29:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:29:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:29:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:29:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:29:01 [DEBUG   ] Finished updating board flags in 1.900 seconds
2015-11-10 02:29:08 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 02:29:28 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:29:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:30:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:30:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:30:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:30:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:30:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:30:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 02:30:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:30:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:30:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:30:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:30:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:30:01 [DEBUG   ] Finished updating board flags in 1.655 seconds
2015-11-10 02:30:08 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:30:21 [DEBUG   ] Got command RPT from MCS: ref #532632
2015-11-10 02:30:21 [DEBUG   ] summary = NORMAL
2015-11-10 02:30:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532632  14 57336  9021419 A NORMALNORMAL'
2015-11-10 02:30:28 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 02:30:48 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:31:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:31:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:31:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:31:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:31:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:31:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:31:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:31:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:31:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:31:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:31:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 02:31:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:31:01 [DEBUG   ] Finished updating board flags in 1.592 seconds
2015-11-10 02:31:08 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:31:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:31:48 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:32:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:32:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:32:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:32:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:32:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:32:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:32:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:32:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:32:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:32:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:32:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 02:32:01 [DEBUG   ] Finished updating board flags in 1.489 seconds
2015-11-10 02:32:08 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:32:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:32:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:32:48 [DEBUG   ] Got command RPT from MCS: ref #532636
2015-11-10 02:32:48 [DEBUG   ] summary = NORMAL
2015-11-10 02:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532636  14 57336  9168975 A NORMALNORMAL'
2015-11-10 02:32:48 [DEBUG   ] Got command RPT from MCS: ref #532637
2015-11-10 02:32:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 02:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532637  48 57336  9168975 A NORMALSystem calibrated and operating normally'
2015-11-10 02:33:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:33:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:33:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:33:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:33:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:33:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:33:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:33:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:33:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:33:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:33:01 [DEBUG   ] Finished updating board flags in 1.610 seconds
2015-11-10 02:33:08 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 02:33:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:33:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:34:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:34:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:34:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:34:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:34:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:34:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:34:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:34:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:34:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:34:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:34:01 [DEBUG   ] Finished updating board flags in 1.634 seconds
2015-11-10 02:34:08 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:34:28 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 02:34:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:35:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:35:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:35:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:35:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:35:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:35:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:35:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:35:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:35:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:35:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:35:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:35:02 [DEBUG   ] Finished updating board flags in 1.642 seconds
2015-11-10 02:35:08 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 02:35:21 [DEBUG   ] Got command RPT from MCS: ref #532654
2015-11-10 02:35:21 [DEBUG   ] summary = NORMAL
2015-11-10 02:35:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532654  14 57336  9321507 A NORMALNORMAL'
2015-11-10 02:35:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:35:48 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:36:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:36:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:36:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:36:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:36:00 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:36:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:36:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:36:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:36:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:36:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:36:02 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:36:02 [DEBUG   ] Finished updating board flags in 1.646 seconds
2015-11-10 02:36:08 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:36:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:36:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:37:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:37:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:37:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:37:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:37:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:37:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:37:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:37:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:37:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:37:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:37:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 02:37:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:37:02 [DEBUG   ] Finished updating board flags in 2.338 seconds
2015-11-10 02:37:08 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 02:37:28 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 02:37:48 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 02:37:48 [DEBUG   ] Got command RPT from MCS: ref #532663
2015-11-10 02:37:48 [DEBUG   ] summary = NORMAL
2015-11-10 02:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532663  14 57336  9468971 A NORMALNORMAL'
2015-11-10 02:37:48 [DEBUG   ] Got command RPT from MCS: ref #532664
2015-11-10 02:37:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 02:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532664  48 57336  9468971 A NORMALSystem calibrated and operating normally'
2015-11-10 02:38:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:38:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:38:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:38:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:38:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:38:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 02:38:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 02:38:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:38:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:38:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:38:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:38:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:38:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 02:38:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:38:02 [DEBUG   ] Finished updating board flags in 2.062 seconds
2015-11-10 02:38:08 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 02:38:28 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-10 02:38:48 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 02:39:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:39:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:39:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:39:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:39:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:39:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:39:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:39:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:39:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:39:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:39:02 [DEBUG   ] Finished updating board flags in 2.022 seconds
2015-11-10 02:39:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 02:39:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 02:39:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 02:40:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:40:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:40:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:40:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:40:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:40:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:40:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:40:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:40:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:40:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:40:02 [DEBUG   ] Finished updating board flags in 1.987 seconds
2015-11-10 02:40:08 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2015-11-10 02:40:21 [DEBUG   ] Got command RPT from MCS: ref #532683
2015-11-10 02:40:21 [DEBUG   ] summary = NORMAL
2015-11-10 02:40:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532683  14 57336  9621583 A NORMALNORMAL'
2015-11-10 02:40:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 02:40:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 02:41:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:41:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:41:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:41:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:41:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:41:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:41:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:41:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:41:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:41:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:41:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 02:41:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:41:02 [DEBUG   ] Finished updating board flags in 1.805 seconds
2015-11-10 02:41:08 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:41:28 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:41:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 02:42:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:42:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:42:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:42:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:42:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:42:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:42:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:42:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:42:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:42:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:42:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:42:02 [DEBUG   ] Finished updating board flags in 1.786 seconds
2015-11-10 02:42:08 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 02:42:28 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:42:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:42:48 [DEBUG   ] Got command RPT from MCS: ref #532687
2015-11-10 02:42:48 [DEBUG   ] summary = NORMAL
2015-11-10 02:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532687  14 57336  9768975 A NORMALNORMAL'
2015-11-10 02:42:48 [DEBUG   ] Got command RPT from MCS: ref #532688
2015-11-10 02:42:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 02:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532688  48 57336  9768975 A NORMALSystem calibrated and operating normally'
2015-11-10 02:43:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:43:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:43:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:43:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:43:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:43:01 [DEBUG   ] Got command STP from MCS: ref #532705
2015-11-10 02:43:01 [INFO    ] Finished processing BAM request
2015-11-10 02:43:01 [INFO    ] Shutdown the adder chain on beam #3
2015-11-10 02:43:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   532705   8 57336  9781291 A NORMAL'
2015-11-10 02:43:01 [DEBUG   ] Got command STP from MCS: ref #532706
2015-11-10 02:43:01 [INFO    ] Finished processing BAM request
2015-11-10 02:43:01 [INFO    ] Shutdown the adder chain on beam #4
2015-11-10 02:43:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   532706   8 57336  9781295 A NORMAL'
2015-11-10 02:43:01 [DEBUG   ] Got command STP from MCS: ref #532704
2015-11-10 02:43:01 [INFO    ] Finished processing BAM request
2015-11-10 02:43:01 [INFO    ] Shutdown the adder chain on beam #2
2015-11-10 02:43:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   532704   8 57336  9781299 A NORMAL'
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 02:43:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 02:43:01 [DEBUG   ] spawn_process: at 2015-11-10 02:43:01 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 02:43:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:43:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:43:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:43:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:43:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:43:02 [DEBUG   ] Finished updating board flags in 1.603 seconds
2015-11-10 02:43:08 [DEBUG   ] Finished updating ant stats in 0.593 seconds
2015-11-10 02:43:29 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-10 02:43:49 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 02:44:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:44:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:44:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:44:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:44:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:44:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:44:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:44:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:44:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:44:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:44:02 [DEBUG   ] Finished updating board flags in 1.698 seconds
2015-11-10 02:44:09 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 02:44:29 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 02:44:49 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 02:45:00 [DEBUG   ] Got command INI from MCS: ref #532711
2015-11-10 02:45:00 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2015-11-10 02:45:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   532711   8 57336  9900472 ABOOTING'
2015-11-10 02:45:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999607
2015-11-10 02:45:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999607
2015-11-10 02:45:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999607
2015-11-10 02:45:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999607
2015-11-10 02:45:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999607
2015-11-10 02:45:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999607
2015-11-10 02:45:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999607
2015-11-10 02:45:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999607
2015-11-10 02:45:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999607
2015-11-10 02:45:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999607
2015-11-10 02:45:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 02:45:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:45:02 [DEBUG   ] Finished updating board flags in 1.609 seconds
2015-11-10 02:45:21 [DEBUG   ] Got command RPT from MCS: ref #532719
2015-11-10 02:45:21 [DEBUG   ] summary = BOOTING
2015-11-10 02:45:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532719  15 57336  9921679 ABOOTINGBOOTING'
2015-11-10 02:45:24 [INFO    ] -------------------
2015-11-10 02:45:24 [INFO    ]  LWA board status
2015-11-10 02:45:24 [INFO    ] -------------------
2015-11-10 02:45:24 [INFO    ] DP 1: 2 lwa27
2015-11-10 02:45:24 [INFO    ] DP 1: 3 lwa24
2015-11-10 02:45:24 [INFO    ] DP 1: 4 lwa23
2015-11-10 02:45:24 [INFO    ] DP 1: 5 lwa18
2015-11-10 02:45:24 [INFO    ] DP 1: 6 lwa10
2015-11-10 02:45:24 [INFO    ] DP 1: 7 lwa32
2015-11-10 02:45:24 [INFO    ] DP 1: 8 lwa11
2015-11-10 02:45:24 [INFO    ] DP 1: 9 lwa30
2015-11-10 02:45:24 [INFO    ] DP 1: 10 lwa12
2015-11-10 02:45:24 [INFO    ] DP 1: 11 lwa8
2015-11-10 02:45:24 [INFO    ] DP 1: 12 lwa13
2015-11-10 02:45:24 [INFO    ] DP 1: 13 lwa26
2015-11-10 02:45:24 [INFO    ] DP 1: 14 lwa14
2015-11-10 02:45:24 [INFO    ] DP 1: 16 lwa22
2015-11-10 02:45:24 [INFO    ] DP 1: 17 lwa20
2015-11-10 02:45:24 [INFO    ] DP 1: 18 lwa1
2015-11-10 02:45:24 [INFO    ] DP 1: 19 lwa5
2015-11-10 02:45:24 [INFO    ] DP 1: 20 lwa25
2015-11-10 02:45:24 [INFO    ] DP 1: 21 lwa35
2015-11-10 02:45:24 [INFO    ] DP 1: 22 lwa19
2015-11-10 02:45:24 [INFO    ] DP 1: 23 lwa21
2015-11-10 02:45:24 [INFO    ] DP 1: 24 lwa6
2015-11-10 02:45:24 [INFO    ] DP 1: 25 lwa7
2015-11-10 02:45:24 [INFO    ] DP 1: 26 lwa33
2015-11-10 02:45:24 [INFO    ] DP 1: 27 lwa15
2015-11-10 02:45:24 [INFO    ] DP 1: 28 lwa28
2015-11-10 02:45:24 [INFO    ] DP 2: 1 lwa16
2015-11-10 02:45:24 [INFO    ] DP 2: 15 lwa34
2015-11-10 02:45:24 [INFO    ] Total of 28 boards
2015-11-10 02:45:24 [INFO    ] Total of 260 stands
2015-11-10 02:45:24 [INFO    ] -------------------
2015-11-10 02:45:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3124
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3125
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3127
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3128
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3129
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3130
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3131
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3132
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3133
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3134
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3135
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3136
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3137
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3138
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3139
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3140
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3141
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3142
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3143
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3144
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3145
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3146
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3147
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3149
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3150
2015-11-10 02:45:31 [DEBUG   ] 2015-11-10 02:45:31 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 3151
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3124 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3125 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3127 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3128 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3129 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3130 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3131 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3132 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3133 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3134 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3135 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3136 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3137 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3138 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3139 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3140 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3141 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3142 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3143 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3144 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3145 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3146 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3147 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3149 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3150 exited with status 0
2015-11-10 02:45:43 [DEBUG   ] 2015-11-10 02:45:43 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 3151 exited with status 0
2015-11-10 02:45:43 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2015-11-10 02:45:43 [DEBUG   ] spawn_process: at 2015-11-10 02:45:43 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:43 [DEBUG   ] spawn_process: at 2015-11-10 02:45:43 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:43 [DEBUG   ] spawn_process: at 2015-11-10 02:45:43 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:43 [DEBUG   ] spawn_process: at 2015-11-10 02:45:43 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] spawn_process: at 2015-11-10 02:45:44 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2015-11-10 02:45:44 [DEBUG   ] 2015-11-10 02:45:44 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 3180
2015-11-10 02:45:44 [DEBUG   ] 2015-11-10 02:45:44 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 3181
2015-11-10 02:45:49 [DEBUG   ] 2015-11-10 02:45:49 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 3180 exited with status 0
2015-11-10 02:45:49 [DEBUG   ] 2015-11-10 02:45:49 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 3181 exited with status 0
2015-11-10 02:45:49 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2015-11-10 02:45:49 [DEBUG   ] spawn_process: at 2015-11-10 02:45:49 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2015-11-10 02:45:49 [DEBUG   ] spawn_process: at 2015-11-10 02:45:49 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2015-11-10 02:45:49 [INFO    ] Setting Beam 1 xbars
2015-11-10 02:45:49 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2015-11-10 02:45:49 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2015-11-10 02:45:49 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2015-11-10 02:45:49 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2015-11-10 02:45:50 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2015-11-10 02:45:50 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2015-11-10 02:45:50 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2015-11-10 02:45:50 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2015-11-10 02:45:50 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2015-11-10 02:45:50 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2015-11-10 02:45:50 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2015-11-10 02:45:50 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2015-11-10 02:45:50 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2015-11-10 02:45:51 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2015-11-10 02:45:51 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2015-11-10 02:45:51 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2015-11-10 02:45:51 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2015-11-10 02:45:51 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2015-11-10 02:45:51 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2015-11-10 02:45:51 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2015-11-10 02:45:51 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2015-11-10 02:45:51 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2015-11-10 02:45:52 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2015-11-10 02:45:52 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2015-11-10 02:45:52 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2015-11-10 02:45:52 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2015-11-10 02:45:52 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2015-11-10 02:45:52 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2015-11-10 02:45:52 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2015-11-10 02:45:52 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2015-11-10 02:45:52 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2015-11-10 02:45:53 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2015-11-10 02:45:53 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2015-11-10 02:45:53 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2015-11-10 02:45:53 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2015-11-10 02:45:53 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2015-11-10 02:45:53 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2015-11-10 02:45:53 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2015-11-10 02:45:53 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2015-11-10 02:45:53 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2015-11-10 02:45:54 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2015-11-10 02:45:54 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2015-11-10 02:45:54 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2015-11-10 02:45:54 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2015-11-10 02:45:54 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2015-11-10 02:45:54 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2015-11-10 02:45:54 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2015-11-10 02:45:54 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2015-11-10 02:45:54 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2015-11-10 02:45:54 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2015-11-10 02:45:55 [INFO    ] Setting Beam 2 xbars
2015-11-10 02:45:55 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2015-11-10 02:45:55 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2015-11-10 02:45:55 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2015-11-10 02:45:55 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2015-11-10 02:45:55 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2015-11-10 02:45:55 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2015-11-10 02:45:55 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2015-11-10 02:45:55 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2015-11-10 02:45:55 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2015-11-10 02:45:56 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2015-11-10 02:45:56 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2015-11-10 02:45:56 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2015-11-10 02:45:56 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2015-11-10 02:45:56 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2015-11-10 02:45:56 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2015-11-10 02:45:56 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2015-11-10 02:45:56 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2015-11-10 02:45:57 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2015-11-10 02:45:57 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2015-11-10 02:45:57 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2015-11-10 02:45:57 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2015-11-10 02:45:57 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2015-11-10 02:45:57 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2015-11-10 02:45:57 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2015-11-10 02:45:58 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2015-11-10 02:45:58 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2015-11-10 02:45:58 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2015-11-10 02:45:58 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2015-11-10 02:45:58 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2015-11-10 02:45:58 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2015-11-10 02:45:58 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2015-11-10 02:45:58 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2015-11-10 02:45:58 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2015-11-10 02:45:59 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2015-11-10 02:45:59 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2015-11-10 02:45:59 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2015-11-10 02:45:59 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2015-11-10 02:45:59 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2015-11-10 02:45:59 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2015-11-10 02:45:59 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2015-11-10 02:45:59 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2015-11-10 02:45:59 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2015-11-10 02:46:00 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2015-11-10 02:46:00 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2015-11-10 02:46:00 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2015-11-10 02:46:00 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2015-11-10 02:46:00 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2015-11-10 02:46:00 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2015-11-10 02:46:00 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2015-11-10 02:46:00 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2015-11-10 02:46:00 [INFO    ] Setting Beam 3 xbars
2015-11-10 02:46:00 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2015-11-10 02:46:01 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2015-11-10 02:46:01 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2015-11-10 02:46:01 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2015-11-10 02:46:01 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2015-11-10 02:46:01 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2015-11-10 02:46:01 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2015-11-10 02:46:01 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2015-11-10 02:46:01 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2015-11-10 02:46:01 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2015-11-10 02:46:02 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2015-11-10 02:46:02 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2015-11-10 02:46:02 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2015-11-10 02:46:02 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2015-11-10 02:46:02 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2015-11-10 02:46:02 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2015-11-10 02:46:02 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2015-11-10 02:46:02 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2015-11-10 02:46:02 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2015-11-10 02:46:03 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2015-11-10 02:46:03 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2015-11-10 02:46:03 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2015-11-10 02:46:03 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2015-11-10 02:46:03 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2015-11-10 02:46:03 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2015-11-10 02:46:03 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2015-11-10 02:46:03 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2015-11-10 02:46:03 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2015-11-10 02:46:04 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2015-11-10 02:46:04 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2015-11-10 02:46:04 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2015-11-10 02:46:04 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2015-11-10 02:46:04 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2015-11-10 02:46:04 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2015-11-10 02:46:04 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2015-11-10 02:46:04 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2015-11-10 02:46:04 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2015-11-10 02:46:05 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2015-11-10 02:46:05 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2015-11-10 02:46:05 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2015-11-10 02:46:05 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2015-11-10 02:46:05 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2015-11-10 02:46:05 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2015-11-10 02:46:05 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2015-11-10 02:46:05 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2015-11-10 02:46:05 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2015-11-10 02:46:06 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2015-11-10 02:46:06 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2015-11-10 02:46:06 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2015-11-10 02:46:06 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2015-11-10 02:46:06 [INFO    ] Setting Beam 4 xbars
2015-11-10 02:46:06 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2015-11-10 02:46:06 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2015-11-10 02:46:06 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2015-11-10 02:46:06 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2015-11-10 02:46:06 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2015-11-10 02:46:06 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2015-11-10 02:46:07 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2015-11-10 02:46:07 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2015-11-10 02:46:07 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2015-11-10 02:46:07 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2015-11-10 02:46:07 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2015-11-10 02:46:07 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2015-11-10 02:46:07 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2015-11-10 02:46:07 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2015-11-10 02:46:07 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2015-11-10 02:46:08 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2015-11-10 02:46:08 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2015-11-10 02:46:08 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2015-11-10 02:46:08 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2015-11-10 02:46:08 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2015-11-10 02:46:08 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2015-11-10 02:46:08 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2015-11-10 02:46:08 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2015-11-10 02:46:08 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2015-11-10 02:46:09 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2015-11-10 02:46:09 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2015-11-10 02:46:09 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2015-11-10 02:46:09 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2015-11-10 02:46:09 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2015-11-10 02:46:09 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2015-11-10 02:46:09 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2015-11-10 02:46:09 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2015-11-10 02:46:09 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2015-11-10 02:46:10 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2015-11-10 02:46:10 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2015-11-10 02:46:10 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2015-11-10 02:46:10 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2015-11-10 02:46:10 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2015-11-10 02:46:10 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2015-11-10 02:46:10 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2015-11-10 02:46:10 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2015-11-10 02:46:10 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2015-11-10 02:46:11 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2015-11-10 02:46:11 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2015-11-10 02:46:11 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2015-11-10 02:46:11 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2015-11-10 02:46:11 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2015-11-10 02:46:11 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2015-11-10 02:46:11 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2015-11-10 02:46:11 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2015-11-10 02:46:11 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2015-11-10 02:46:12 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2015-11-10 02:46:12 [INFO    ] All xbars set
2015-11-10 02:46:12 [INFO    ] Setting adder chain routing for all beams
2015-11-10 02:46:16 [INFO    ] Setting beam ids
2015-11-10 02:46:27 [INFO    ] Starting beam delay calibration
2015-11-10 02:46:39 [INFO    ] Beam 1 latency is 5725
2015-11-10 02:46:39 [INFO    ] Beam 2 latency is 5740
2015-11-10 02:46:39 [INFO    ] Beam 3 latency is 5738
2015-11-10 02:46:39 [INFO    ] Beam 4 latency is 5728
2015-11-10 02:46:39 [INFO    ] Stop calibration
2015-11-10 02:46:47 [INFO    ] Beam delay calibration successful
2015-11-10 02:46:47 [INFO    ] Calibration passed
2015-11-10 02:46:47 [DEBUG   ] spawn_process: at 2015-11-10 02:46:47 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2015-11-10 02:46:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2015-11-10 02:46:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2015-11-10 02:46:47 [DEBUG   ] spawn_process: at 2015-11-10 02:46:47 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2015-11-10 02:46:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2015-11-10 02:46:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2015-11-10 02:46:47 [DEBUG   ] spawn_process: at 2015-11-10 02:46:47 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2015-11-10 02:46:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2015-11-10 02:46:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2015-11-10 02:46:47 [DEBUG   ] spawn_process: at 2015-11-10 02:46:47 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2015-11-10 02:46:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2015-11-10 02:46:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2015-11-10 02:46:47 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -4 msec
2015-11-10 02:46:47 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 443 ms (1447123607.446 s) with PID 4744 and time 1447123608
2015-11-10 02:46:47 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -3 msec
2015-11-10 02:46:47 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 531 ms (1447123607.535 s) with PID 4747 and time 1447123608
2015-11-10 02:46:48 [DEBUG   ] Finished updating ant stats in 0.714 seconds
2015-11-10 02:46:48 [INFO    ] Started antennas statistics background thread
2015-11-10 02:46:48 [INFO    ] Started board flags background thread
2015-11-10 02:46:48 [INFO    ] Started BAM background thread
2015-11-10 02:46:48 [INFO    ] Finished the INI process (ref. #532711) in 108.186 s
2015-11-10 02:46:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 02:46:49 [DEBUG   ] Finished updating board flags in 1.356 seconds
2015-11-10 02:47:08 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:47:28 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:47:30 [DEBUG   ] Got command RPT from MCS: ref #532725
2015-11-10 02:47:30 [DEBUG   ] summary = NORMAL
2015-11-10 02:47:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532725  14 57336 10050467 A NORMALNORMAL'
2015-11-10 02:47:30 [DEBUG   ] Got command RPT from MCS: ref #532726
2015-11-10 02:47:30 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 02:47:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532726  48 57336 10050467 A NORMALSystem calibrated and operating normally'
2015-11-10 02:47:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:47:48 [DEBUG   ] Got command RPT from MCS: ref #532730
2015-11-10 02:47:48 [DEBUG   ] summary = NORMAL
2015-11-10 02:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532730  14 57336 10068971 A NORMALNORMAL'
2015-11-10 02:47:48 [DEBUG   ] Got command RPT from MCS: ref #532731
2015-11-10 02:47:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 02:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532731  48 57336 10068972 A NORMALSystem calibrated and operating normally'
2015-11-10 02:47:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:47:49 [DEBUG   ] Finished updating board flags in 1.216 seconds
2015-11-10 02:48:08 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:48:28 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:48:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:48:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 02:48:50 [DEBUG   ] Finished updating board flags in 1.739 seconds
2015-11-10 02:49:08 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:49:28 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:49:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:49:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:49:49 [DEBUG   ] Finished updating board flags in 1.066 seconds
2015-11-10 02:50:08 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:50:21 [DEBUG   ] Got command RPT from MCS: ref #532747
2015-11-10 02:50:21 [DEBUG   ] summary = NORMAL
2015-11-10 02:50:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532747  14 57336 10221791 A NORMALNORMAL'
2015-11-10 02:50:28 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:50:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:50:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 02:50:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:50:50 [DEBUG   ] Finished updating board flags in 1.606 seconds
2015-11-10 02:51:08 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:51:28 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 02:51:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 02:51:50 [DEBUG   ] Finished updating board flags in 1.524 seconds
2015-11-10 02:52:08 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2015-11-10 02:52:28 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:52:48 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:52:48 [DEBUG   ] Got command RPT from MCS: ref #532751
2015-11-10 02:52:48 [DEBUG   ] summary = NORMAL
2015-11-10 02:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532751  14 57336 10368975 A NORMALNORMAL'
2015-11-10 02:52:48 [DEBUG   ] Got command RPT from MCS: ref #532752
2015-11-10 02:52:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 02:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532752  48 57336 10368975 A NORMALSystem calibrated and operating normally'
2015-11-10 02:52:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 02:52:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 02:52:50 [DEBUG   ] Finished updating board flags in 1.411 seconds
2015-11-10 02:53:08 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:53:28 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:53:48 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:53:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 02:53:50 [DEBUG   ] Finished updating board flags in 1.411 seconds
2015-11-10 02:54:08 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:54:28 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:54:48 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:54:50 [DEBUG   ] Finished updating board flags in 1.392 seconds
2015-11-10 02:55:08 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:55:21 [DEBUG   ] Got command RPT from MCS: ref #532772
2015-11-10 02:55:21 [DEBUG   ] summary = NORMAL
2015-11-10 02:55:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532772  14 57336 10521895 A NORMALNORMAL'
2015-11-10 02:55:28 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:55:48 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:55:50 [DEBUG   ] Finished updating board flags in 1.204 seconds
2015-11-10 02:56:08 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:56:28 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:56:48 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:56:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 02:56:50 [DEBUG   ] Finished updating board flags in 1.258 seconds
2015-11-10 02:57:08 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:57:28 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:57:48 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:57:48 [DEBUG   ] Got command RPT from MCS: ref #532781
2015-11-10 02:57:48 [DEBUG   ] summary = NORMAL
2015-11-10 02:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532781  14 57336 10668975 A NORMALNORMAL'
2015-11-10 02:57:48 [DEBUG   ] Got command RPT from MCS: ref #532782
2015-11-10 02:57:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 02:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532782  48 57336 10668975 A NORMALSystem calibrated and operating normally'
2015-11-10 02:57:50 [DEBUG   ] Finished updating board flags in 1.135 seconds
2015-11-10 02:58:08 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:58:28 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:58:48 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 02:58:50 [DEBUG   ] Finished updating board flags in 1.024 seconds
2015-11-10 02:59:08 [DEBUG   ] Finished updating ant stats in 0.397 seconds
2015-11-10 02:59:29 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:59:49 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 02:59:51 [DEBUG   ] Finished updating board flags in 1.991 seconds
2015-11-10 03:00:09 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 03:00:22 [DEBUG   ] Got command RPT from MCS: ref #532801
2015-11-10 03:00:22 [DEBUG   ] summary = NORMAL
2015-11-10 03:00:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532801  14 57336 10822015 A NORMALNORMAL'
2015-11-10 03:00:29 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 03:00:49 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 03:00:51 [DEBUG   ] Finished updating board flags in 1.783 seconds
2015-11-10 03:01:09 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 03:01:29 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:01:49 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 03:01:50 [DEBUG   ] Finished updating board flags in 1.071 seconds
2015-11-10 03:02:09 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:02:29 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 03:02:48 [DEBUG   ] Got command RPT from MCS: ref #532805
2015-11-10 03:02:48 [DEBUG   ] summary = NORMAL
2015-11-10 03:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532805  14 57336 10968983 A NORMALNORMAL'
2015-11-10 03:02:48 [DEBUG   ] Got command RPT from MCS: ref #532806
2015-11-10 03:02:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 03:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532806  48 57336 10968983 A NORMALSystem calibrated and operating normally'
2015-11-10 03:02:49 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 03:02:50 [DEBUG   ] Finished updating board flags in 1.016 seconds
2015-11-10 03:03:09 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 03:03:29 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:03:49 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:03:50 [DEBUG   ] Finished updating board flags in 0.973 seconds
2015-11-10 03:04:09 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:04:29 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:04:49 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:04:51 [DEBUG   ] Finished updating board flags in 1.397 seconds
2015-11-10 03:05:09 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:05:22 [DEBUG   ] Got command RPT from MCS: ref #532823
2015-11-10 03:05:22 [DEBUG   ] summary = NORMAL
2015-11-10 03:05:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532823  14 57336 11122120 A NORMALNORMAL'
2015-11-10 03:05:29 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-10 03:05:49 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 03:05:51 [DEBUG   ] Finished updating board flags in 1.147 seconds
2015-11-10 03:06:09 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-10 03:06:29 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 03:06:49 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-10 03:06:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-10 03:06:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-10 03:06:51 [DEBUG   ] Finished updating board flags in 1.031 seconds
2015-11-10 03:07:09 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:07:29 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:07:48 [DEBUG   ] Got command RPT from MCS: ref #532832
2015-11-10 03:07:48 [DEBUG   ] summary = NORMAL
2015-11-10 03:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532832  14 57336 11268987 A NORMALNORMAL'
2015-11-10 03:07:48 [DEBUG   ] Got command RPT from MCS: ref #532833
2015-11-10 03:07:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 03:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532833  48 57336 11268987 A NORMALSystem calibrated and operating normally'
2015-11-10 03:07:49 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-10 03:07:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-10 03:07:52 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-10 03:07:52 [DEBUG   ] Finished updating board flags in 2.017 seconds
2015-11-10 03:08:09 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 03:08:29 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:08:49 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 03:08:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 03:08:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 03:08:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 03:08:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 03:08:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 03:08:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 03:08:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 03:08:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 03:08:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 03:08:51 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 03:08:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 03:08:51 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 03:08:51 [DEBUG   ] Finished updating board flags in 1.402 seconds
2015-11-10 03:09:00 [DEBUG   ] Got command INI from MCS: ref #532852
2015-11-10 03:09:00 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2015-11-10 03:09:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   532852   8 57336 11340476 ABOOTING'
2015-11-10 03:09:20 [INFO    ] -------------------
2015-11-10 03:09:20 [INFO    ]  LWA board status
2015-11-10 03:09:20 [INFO    ] -------------------
2015-11-10 03:09:20 [INFO    ] DP 1: 2 lwa27
2015-11-10 03:09:20 [INFO    ] DP 1: 3 lwa24
2015-11-10 03:09:20 [INFO    ] DP 1: 4 lwa23
2015-11-10 03:09:20 [INFO    ] DP 1: 5 lwa18
2015-11-10 03:09:20 [INFO    ] DP 1: 6 lwa10
2015-11-10 03:09:20 [INFO    ] DP 1: 7 lwa32
2015-11-10 03:09:20 [INFO    ] DP 1: 8 lwa11
2015-11-10 03:09:20 [INFO    ] DP 1: 9 lwa30
2015-11-10 03:09:20 [INFO    ] DP 1: 10 lwa12
2015-11-10 03:09:20 [INFO    ] DP 1: 11 lwa8
2015-11-10 03:09:20 [INFO    ] DP 1: 12 lwa13
2015-11-10 03:09:20 [INFO    ] DP 1: 13 lwa26
2015-11-10 03:09:20 [INFO    ] DP 1: 14 lwa14
2015-11-10 03:09:20 [INFO    ] DP 1: 16 lwa22
2015-11-10 03:09:20 [INFO    ] DP 1: 17 lwa20
2015-11-10 03:09:20 [INFO    ] DP 1: 18 lwa1
2015-11-10 03:09:20 [INFO    ] DP 1: 19 lwa5
2015-11-10 03:09:20 [INFO    ] DP 1: 20 lwa25
2015-11-10 03:09:20 [INFO    ] DP 1: 21 lwa35
2015-11-10 03:09:20 [INFO    ] DP 1: 22 lwa19
2015-11-10 03:09:20 [INFO    ] DP 1: 23 lwa21
2015-11-10 03:09:20 [INFO    ] DP 1: 24 lwa6
2015-11-10 03:09:20 [INFO    ] DP 1: 25 lwa7
2015-11-10 03:09:20 [INFO    ] DP 1: 26 lwa33
2015-11-10 03:09:20 [INFO    ] DP 1: 27 lwa15
2015-11-10 03:09:20 [INFO    ] DP 1: 28 lwa28
2015-11-10 03:09:20 [INFO    ] DP 2: 1 lwa16
2015-11-10 03:09:20 [INFO    ] DP 2: 15 lwa34
2015-11-10 03:09:20 [INFO    ] Total of 28 boards
2015-11-10 03:09:20 [INFO    ] Total of 260 stands
2015-11-10 03:09:20 [INFO    ] -------------------
2015-11-10 03:09:20 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7590
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7591
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7592
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7593
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7594
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7595
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7596
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7597
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7598
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7599
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7600
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7601
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7602
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7603
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7604
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7605
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7606
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7607
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7608
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7609
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7610
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7611
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7612
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7613
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7614
2015-11-10 03:09:27 [DEBUG   ] 2015-11-10 03:09:27 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 7615
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7590 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7591 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7592 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7593 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7594 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7595 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7596 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7597 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7598 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7599 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7600 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7601 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7602 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7603 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7604 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7605 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7606 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7607 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7608 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7609 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7610 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7611 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7612 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7613 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7614 exited with status 0
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 7615 exited with status 0
2015-11-10 03:09:39 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] spawn_process: at 2015-11-10 03:09:39 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 7642
2015-11-10 03:09:39 [DEBUG   ] 2015-11-10 03:09:39 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 7643
2015-11-10 03:09:45 [DEBUG   ] 2015-11-10 03:09:45 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 7642 exited with status 0
2015-11-10 03:09:45 [DEBUG   ] 2015-11-10 03:09:45 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 7643 exited with status 0
2015-11-10 03:09:45 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2015-11-10 03:09:45 [DEBUG   ] spawn_process: at 2015-11-10 03:09:45 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2015-11-10 03:09:45 [DEBUG   ] spawn_process: at 2015-11-10 03:09:45 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2015-11-10 03:09:45 [INFO    ] Setting Beam 1 xbars
2015-11-10 03:09:45 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2015-11-10 03:09:45 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2015-11-10 03:09:45 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2015-11-10 03:09:45 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2015-11-10 03:09:45 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2015-11-10 03:09:46 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2015-11-10 03:09:46 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2015-11-10 03:09:46 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2015-11-10 03:09:46 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2015-11-10 03:09:46 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2015-11-10 03:09:46 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2015-11-10 03:09:46 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2015-11-10 03:09:46 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2015-11-10 03:09:46 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2015-11-10 03:09:47 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2015-11-10 03:09:47 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2015-11-10 03:09:47 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2015-11-10 03:09:47 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2015-11-10 03:09:47 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2015-11-10 03:09:47 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2015-11-10 03:09:47 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2015-11-10 03:09:47 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2015-11-10 03:09:47 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2015-11-10 03:09:48 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2015-11-10 03:09:48 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2015-11-10 03:09:48 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2015-11-10 03:09:48 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2015-11-10 03:09:48 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2015-11-10 03:09:48 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2015-11-10 03:09:48 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2015-11-10 03:09:48 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2015-11-10 03:09:48 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2015-11-10 03:09:49 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2015-11-10 03:09:49 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2015-11-10 03:09:49 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2015-11-10 03:09:49 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2015-11-10 03:09:49 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2015-11-10 03:09:49 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2015-11-10 03:09:49 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2015-11-10 03:09:49 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2015-11-10 03:09:49 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2015-11-10 03:09:50 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2015-11-10 03:09:50 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2015-11-10 03:09:50 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2015-11-10 03:09:50 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2015-11-10 03:09:50 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2015-11-10 03:09:50 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2015-11-10 03:09:50 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2015-11-10 03:09:50 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2015-11-10 03:09:50 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2015-11-10 03:09:51 [INFO    ] Setting Beam 2 xbars
2015-11-10 03:09:51 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2015-11-10 03:09:51 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2015-11-10 03:09:51 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2015-11-10 03:09:51 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2015-11-10 03:09:51 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2015-11-10 03:09:51 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2015-11-10 03:09:51 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2015-11-10 03:09:51 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2015-11-10 03:09:51 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2015-11-10 03:09:51 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2015-11-10 03:09:52 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2015-11-10 03:09:52 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2015-11-10 03:09:52 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2015-11-10 03:09:52 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2015-11-10 03:09:52 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2015-11-10 03:09:52 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2015-11-10 03:09:52 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2015-11-10 03:09:52 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2015-11-10 03:09:52 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2015-11-10 03:09:53 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2015-11-10 03:09:53 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2015-11-10 03:09:53 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2015-11-10 03:09:53 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2015-11-10 03:09:53 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2015-11-10 03:09:53 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2015-11-10 03:09:53 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2015-11-10 03:09:53 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2015-11-10 03:09:53 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2015-11-10 03:09:54 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2015-11-10 03:09:54 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2015-11-10 03:09:54 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2015-11-10 03:09:54 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2015-11-10 03:09:54 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2015-11-10 03:09:54 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2015-11-10 03:09:54 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2015-11-10 03:09:54 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2015-11-10 03:09:54 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2015-11-10 03:09:55 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2015-11-10 03:09:55 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2015-11-10 03:09:55 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2015-11-10 03:09:55 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2015-11-10 03:09:55 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2015-11-10 03:09:55 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2015-11-10 03:09:55 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2015-11-10 03:09:55 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2015-11-10 03:09:55 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2015-11-10 03:09:56 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2015-11-10 03:09:56 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2015-11-10 03:09:56 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2015-11-10 03:09:56 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2015-11-10 03:09:56 [INFO    ] Setting Beam 3 xbars
2015-11-10 03:09:56 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2015-11-10 03:09:56 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2015-11-10 03:09:56 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2015-11-10 03:09:56 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2015-11-10 03:09:56 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2015-11-10 03:09:57 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2015-11-10 03:09:57 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2015-11-10 03:09:57 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2015-11-10 03:09:57 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2015-11-10 03:09:57 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2015-11-10 03:09:57 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2015-11-10 03:09:57 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2015-11-10 03:09:57 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2015-11-10 03:09:57 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2015-11-10 03:09:58 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2015-11-10 03:09:58 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2015-11-10 03:09:58 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2015-11-10 03:09:58 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2015-11-10 03:09:58 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2015-11-10 03:09:58 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2015-11-10 03:09:58 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2015-11-10 03:09:58 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2015-11-10 03:09:58 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2015-11-10 03:09:59 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2015-11-10 03:09:59 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2015-11-10 03:09:59 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2015-11-10 03:09:59 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2015-11-10 03:09:59 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2015-11-10 03:09:59 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2015-11-10 03:09:59 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2015-11-10 03:09:59 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2015-11-10 03:09:59 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2015-11-10 03:09:59 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2015-11-10 03:10:00 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2015-11-10 03:10:00 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2015-11-10 03:10:00 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2015-11-10 03:10:00 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2015-11-10 03:10:00 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2015-11-10 03:10:00 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2015-11-10 03:10:00 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2015-11-10 03:10:00 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2015-11-10 03:10:00 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2015-11-10 03:10:01 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2015-11-10 03:10:01 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2015-11-10 03:10:01 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2015-11-10 03:10:01 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2015-11-10 03:10:01 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2015-11-10 03:10:01 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2015-11-10 03:10:01 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2015-11-10 03:10:01 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2015-11-10 03:10:01 [INFO    ] Setting Beam 4 xbars
2015-11-10 03:10:01 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2015-11-10 03:10:02 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2015-11-10 03:10:02 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2015-11-10 03:10:02 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2015-11-10 03:10:02 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2015-11-10 03:10:02 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2015-11-10 03:10:02 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2015-11-10 03:10:02 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2015-11-10 03:10:02 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2015-11-10 03:10:02 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2015-11-10 03:10:03 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2015-11-10 03:10:03 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2015-11-10 03:10:03 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2015-11-10 03:10:03 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2015-11-10 03:10:03 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2015-11-10 03:10:03 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2015-11-10 03:10:03 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2015-11-10 03:10:03 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2015-11-10 03:10:03 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2015-11-10 03:10:04 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2015-11-10 03:10:04 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2015-11-10 03:10:04 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2015-11-10 03:10:04 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2015-11-10 03:10:04 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2015-11-10 03:10:04 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2015-11-10 03:10:04 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2015-11-10 03:10:04 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2015-11-10 03:10:04 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2015-11-10 03:10:05 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2015-11-10 03:10:05 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2015-11-10 03:10:05 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2015-11-10 03:10:05 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2015-11-10 03:10:05 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2015-11-10 03:10:05 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2015-11-10 03:10:05 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2015-11-10 03:10:05 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2015-11-10 03:10:05 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2015-11-10 03:10:06 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2015-11-10 03:10:06 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2015-11-10 03:10:06 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2015-11-10 03:10:06 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2015-11-10 03:10:06 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2015-11-10 03:10:06 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2015-11-10 03:10:06 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2015-11-10 03:10:06 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2015-11-10 03:10:06 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2015-11-10 03:10:07 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2015-11-10 03:10:07 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2015-11-10 03:10:07 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2015-11-10 03:10:07 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2015-11-10 03:10:07 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2015-11-10 03:10:07 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2015-11-10 03:10:07 [INFO    ] All xbars set
2015-11-10 03:10:07 [INFO    ] Setting adder chain routing for all beams
2015-11-10 03:10:12 [INFO    ] Setting beam ids
2015-11-10 03:10:22 [DEBUG   ] Got command RPT from MCS: ref #532853
2015-11-10 03:10:22 [DEBUG   ] summary = BOOTING
2015-11-10 03:10:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532853  15 57336 11422211 ABOOTINGBOOTING'
2015-11-10 03:10:22 [INFO    ] Starting beam delay calibration
2015-11-10 03:10:34 [INFO    ] Beam 1 latency is 5730
2015-11-10 03:10:34 [INFO    ] Beam 2 latency is 5753
2015-11-10 03:10:34 [INFO    ] Beam 3 latency is 5742
2015-11-10 03:10:34 [INFO    ] Beam 4 latency is 5730
2015-11-10 03:10:34 [INFO    ] Stop calibration
2015-11-10 03:10:42 [INFO    ] Beam delay calibration successful
2015-11-10 03:10:42 [INFO    ] Calibration passed
2015-11-10 03:10:42 [DEBUG   ] spawn_process: at 2015-11-10 03:10:42 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2015-11-10 03:10:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2015-11-10 03:10:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2015-11-10 03:10:42 [DEBUG   ] spawn_process: at 2015-11-10 03:10:42 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2015-11-10 03:10:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2015-11-10 03:10:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2015-11-10 03:10:43 [DEBUG   ] spawn_process: at 2015-11-10 03:10:43 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2015-11-10 03:10:43 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2015-11-10 03:10:43 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2015-11-10 03:10:43 [DEBUG   ] spawn_process: at 2015-11-10 03:10:43 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2015-11-10 03:10:43 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2015-11-10 03:10:43 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2015-11-10 03:10:43 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -3 msec
2015-11-10 03:10:43 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 264 ms (1447125043.267 s) with PID 9221 and time 1447125044
2015-11-10 03:10:43 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -3 msec
2015-11-10 03:10:43 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 354 ms (1447125043.357 s) with PID 9224 and time 1447125044
2015-11-10 03:10:44 [DEBUG   ] Finished updating ant stats in 0.891 seconds
2015-11-10 03:10:44 [INFO    ] Started antennas statistics background thread
2015-11-10 03:10:44 [INFO    ] Started board flags background thread
2015-11-10 03:10:44 [INFO    ] Started BAM background thread
2015-11-10 03:10:44 [INFO    ] Finished the INI process (ref. #532852) in 104.005 s
2015-11-10 03:10:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 03:10:45 [DEBUG   ] Finished updating board flags in 1.077 seconds
2015-11-10 03:11:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:11:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:11:30 [DEBUG   ] Got command RPT from MCS: ref #532854
2015-11-10 03:11:30 [DEBUG   ] summary = NORMAL
2015-11-10 03:11:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532854  14 57336 11490475 A NORMALNORMAL'
2015-11-10 03:11:30 [DEBUG   ] Got command RPT from MCS: ref #532855
2015-11-10 03:11:30 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 03:11:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532855  48 57336 11490475 A NORMALSystem calibrated and operating normally'
2015-11-10 03:11:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:11:45 [DEBUG   ] Finished updating board flags in 1.012 seconds
2015-11-10 03:12:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 03:12:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 03:12:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:12:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 03:12:45 [DEBUG   ] Finished updating board flags in 1.031 seconds
2015-11-10 03:12:48 [DEBUG   ] Got command RPT from MCS: ref #532859
2015-11-10 03:12:48 [DEBUG   ] summary = NORMAL
2015-11-10 03:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532859  14 57336 11568979 A NORMALNORMAL'
2015-11-10 03:12:48 [DEBUG   ] Got command RPT from MCS: ref #532860
2015-11-10 03:12:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 03:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532860  48 57336 11568979 A NORMALSystem calibrated and operating normally'
2015-11-10 03:13:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:13:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:13:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:13:46 [DEBUG   ] Finished updating board flags in 2.031 seconds
2015-11-10 03:14:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:14:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:14:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:14:45 [DEBUG   ] Finished updating board flags in 1.289 seconds
2015-11-10 03:15:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:15:22 [DEBUG   ] Got command RPT from MCS: ref #532885
2015-11-10 03:15:22 [DEBUG   ] summary = NORMAL
2015-11-10 03:15:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532885  14 57336 11722311 A NORMALNORMAL'
2015-11-10 03:15:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:15:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:15:46 [DEBUG   ] Finished updating board flags in 1.756 seconds
2015-11-10 03:16:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:16:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:16:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:16:45 [DEBUG   ] Finished updating board flags in 1.018 seconds
2015-11-10 03:17:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 03:17:24 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 03:17:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:17:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 03:17:45 [DEBUG   ] Finished updating board flags in 1.031 seconds
2015-11-10 03:17:48 [DEBUG   ] Got command RPT from MCS: ref #532894
2015-11-10 03:17:48 [DEBUG   ] summary = NORMAL
2015-11-10 03:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532894  14 57336 11868983 A NORMALNORMAL'
2015-11-10 03:17:48 [DEBUG   ] Got command RPT from MCS: ref #532895
2015-11-10 03:17:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 03:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532895  48 57336 11868983 A NORMALSystem calibrated and operating normally'
2015-11-10 03:18:04 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 03:18:24 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:18:44 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:18:45 [DEBUG   ] Finished updating board flags in 0.977 seconds
2015-11-10 03:19:04 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:19:24 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:19:44 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:19:46 [DEBUG   ] Finished updating board flags in 1.393 seconds
2015-11-10 03:20:04 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:20:22 [DEBUG   ] Got command RPT from MCS: ref #532911
2015-11-10 03:20:22 [DEBUG   ] summary = NORMAL
2015-11-10 03:20:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532911  14 57336 12022395 A NORMALNORMAL'
2015-11-10 03:20:24 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:20:44 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:20:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 03:20:46 [DEBUG   ] Finished updating board flags in 1.095 seconds
2015-11-10 03:21:04 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:21:24 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:21:44 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:21:47 [DEBUG   ] Finished updating board flags in 2.018 seconds
2015-11-10 03:22:04 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:22:24 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:22:44 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:22:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 03:22:46 [DEBUG   ] Finished updating board flags in 1.444 seconds
2015-11-10 03:22:48 [DEBUG   ] Got command RPT from MCS: ref #532915
2015-11-10 03:22:48 [DEBUG   ] summary = NORMAL
2015-11-10 03:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532915  14 57336 12168983 A NORMALNORMAL'
2015-11-10 03:22:48 [DEBUG   ] Got command RPT from MCS: ref #532916
2015-11-10 03:22:48 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 03:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532916  48 57336 12168983 A NORMALSystem calibrated and operating normally'
2015-11-10 03:23:04 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:23:24 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:23:44 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:23:46 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2015-11-10 03:23:46 [CRITICAL] MGT underflow detected on beam(s) 1
2015-11-10 03:23:46 [DEBUG   ] Finished updating board flags in 1.417 seconds
2015-11-10 03:24:04 [DEBUG   ] Finished updating ant stats in 0.397 seconds
2015-11-10 03:24:25 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:24:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:24:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-10 03:24:46 [DEBUG   ] Finished updating board flags in 1.405 seconds
2015-11-10 03:25:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:25:22 [DEBUG   ] Got command RPT from MCS: ref #532936
2015-11-10 03:25:22 [DEBUG   ] summary = ERROR
2015-11-10 03:25:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532936  13 57336 12322487 A  ERRORERROR'
2015-11-10 03:25:22 [DEBUG   ] Got command RPT from MCS: ref #532937
2015-11-10 03:25:22 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:25:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532937  77 57336 12322679 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:25:22 [DEBUG   ] Got command RPT from MCS: ref #532938
2015-11-10 03:25:22 [DEBUG   ] summary = ERROR
2015-11-10 03:25:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532938  13 57336 12322883 A  ERRORERROR'
2015-11-10 03:25:23 [DEBUG   ] Got command RPT from MCS: ref #532939
2015-11-10 03:25:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:25:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532939  77 57336 12323087 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:25:23 [DEBUG   ] Got command RPT from MCS: ref #532940
2015-11-10 03:25:23 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 03:25:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532940  12 57336 12323291 A  ERRORBŠÌÍ'
2015-11-10 03:25:25 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:25:45 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 03:25:47 [DEBUG   ] Finished updating board flags in 1.956 seconds
2015-11-10 03:26:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:26:25 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:26:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:26:47 [DEBUG   ] Finished updating board flags in 1.811 seconds
2015-11-10 03:27:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:27:25 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:27:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:27:46 [DEBUG   ] Finished updating board flags in 1.227 seconds
2015-11-10 03:27:48 [DEBUG   ] Got command RPT from MCS: ref #532949
2015-11-10 03:27:48 [DEBUG   ] summary = ERROR
2015-11-10 03:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532949  13 57336 12468983 A  ERRORERROR'
2015-11-10 03:27:48 [DEBUG   ] Got command RPT from MCS: ref #532950
2015-11-10 03:27:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532950  77 57336 12468983 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:28:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:28:25 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:28:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:28:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 03:28:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 03:28:47 [DEBUG   ] Finished updating board flags in 1.749 seconds
2015-11-10 03:29:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:29:25 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:29:45 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 03:29:47 [DEBUG   ] Finished updating board flags in 1.580 seconds
2015-11-10 03:29:58 [DEBUG   ] Got command DRX from MCS: ref #532971
2015-11-10 03:29:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:29:58 [DEBUG   ] run_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:29:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   532971   8 57336 12598007 A  ERROR'
2015-11-10 03:29:58 [INFO    ] Finished starting DRX
2015-11-10 03:29:58 [DEBUG   ] Got command DRX from MCS: ref #532972
2015-11-10 03:29:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:29:58 [DEBUG   ] run_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:29:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   532972   8 57336 12598191 A  ERROR'
2015-11-10 03:29:58 [DEBUG   ] Got command BAM from MCS: ref #532973
2015-11-10 03:29:58 [INFO    ] Finished processing BAM request
2015-11-10 03:29:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   532973   8 57336 12598195 A  ERROR'
2015-11-10 03:29:58 [INFO    ] Finished starting DRX
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 03:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 03:29:58 [DEBUG   ] spawn_process: at 2015-11-10 03:29:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 03:30:05 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-10 03:30:22 [DEBUG   ] Got command RPT from MCS: ref #532975
2015-11-10 03:30:22 [DEBUG   ] summary = ERROR
2015-11-10 03:30:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532975  13 57336 12622575 A  ERRORERROR'
2015-11-10 03:30:22 [DEBUG   ] Got command RPT from MCS: ref #532976
2015-11-10 03:30:22 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:30:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532976  77 57336 12622767 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:30:22 [DEBUG   ] Got command RPT from MCS: ref #532977
2015-11-10 03:30:22 [DEBUG   ] summary = ERROR
2015-11-10 03:30:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532977  13 57336 12622971 A  ERRORERROR'
2015-11-10 03:30:23 [DEBUG   ] Got command RPT from MCS: ref #532978
2015-11-10 03:30:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:30:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532978  77 57336 12623175 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:30:23 [DEBUG   ] Got command RPT from MCS: ref #532979
2015-11-10 03:30:23 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 03:30:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532979  12 57336 12623379 A  ERRORB‹ÌÍ'
2015-11-10 03:30:25 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-10 03:30:38 [DEBUG   ] Got command BAM from MCS: ref #532980
2015-11-10 03:30:38 [INFO    ] Finished processing BAM request
2015-11-10 03:30:38 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:30:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   532980   8 57336 12638035 A  ERROR'
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 03:30:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 03:30:38 [DEBUG   ] spawn_process: at 2015-11-10 03:30:38 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 03:30:45 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 03:30:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 03:30:47 [DEBUG   ] Finished updating board flags in 1.393 seconds
2015-11-10 03:31:05 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:31:18 [DEBUG   ] Got command BAM from MCS: ref #532981
2015-11-10 03:31:18 [INFO    ] Finished processing BAM request
2015-11-10 03:31:18 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:31:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   532981   8 57336 12678027 A  ERROR'
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 03:31:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 03:31:18 [DEBUG   ] spawn_process: at 2015-11-10 03:31:18 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 03:31:25 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:31:45 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:31:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 03:31:47 [DEBUG   ] Finished updating board flags in 1.196 seconds
2015-11-10 03:31:58 [DEBUG   ] Got command BAM from MCS: ref #532982
2015-11-10 03:31:58 [INFO    ] Finished processing BAM request
2015-11-10 03:31:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   532982   8 57336 12718027 A  ERROR'
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 03:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 03:31:58 [DEBUG   ] spawn_process: at 2015-11-10 03:31:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 03:32:05 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:32:25 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:32:38 [DEBUG   ] Got command BAM from MCS: ref #532986
2015-11-10 03:32:38 [INFO    ] Finished processing BAM request
2015-11-10 03:32:38 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:32:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   532986   8 57336 12758031 A  ERROR'
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 03:32:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 03:32:38 [DEBUG   ] spawn_process: at 2015-11-10 03:32:38 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 03:32:45 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:32:47 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 03:32:47 [DEBUG   ] Finished updating board flags in 1.015 seconds
2015-11-10 03:32:48 [DEBUG   ] Got command RPT from MCS: ref #532987
2015-11-10 03:32:48 [DEBUG   ] summary = ERROR
2015-11-10 03:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532987  13 57336 12768983 A  ERRORERROR'
2015-11-10 03:32:48 [DEBUG   ] Got command RPT from MCS: ref #532988
2015-11-10 03:32:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   532988  77 57336 12768983 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:33:05 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 03:33:18 [DEBUG   ] Got command BAM from MCS: ref #533004
2015-11-10 03:33:18 [INFO    ] Finished processing BAM request
2015-11-10 03:33:18 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:33:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533004   8 57336 12798027 A  ERROR'
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 03:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 03:33:18 [DEBUG   ] spawn_process: at 2015-11-10 03:33:18 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 03:33:25 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:33:45 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 03:33:47 [DEBUG   ] Finished updating board flags in 1.001 seconds
2015-11-10 03:33:58 [DEBUG   ] Got command DRX from MCS: ref #533005
2015-11-10 03:33:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:33:58 [DEBUG   ] run_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:33:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533005   8 57336 12838007 A  ERROR'
2015-11-10 03:33:58 [INFO    ] Finished starting DRX
2015-11-10 03:33:58 [DEBUG   ] Got command DRX from MCS: ref #533006
2015-11-10 03:33:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:33:58 [DEBUG   ] run_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:33:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533006   8 57336 12838167 A  ERROR'
2015-11-10 03:33:58 [DEBUG   ] Got command BAM from MCS: ref #533007
2015-11-10 03:33:58 [INFO    ] Finished processing BAM request
2015-11-10 03:33:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533007   8 57336 12838171 A  ERROR'
2015-11-10 03:33:58 [INFO    ] Finished starting DRX
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 03:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 03:33:58 [DEBUG   ] spawn_process: at 2015-11-10 03:33:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 03:34:05 [DEBUG   ] Finished updating ant stats in 0.397 seconds
2015-11-10 03:34:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:34:46 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:34:47 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 03:34:47 [DEBUG   ] Finished updating board flags in 0.994 seconds
2015-11-10 03:35:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:35:22 [DEBUG   ] Got command RPT from MCS: ref #533010
2015-11-10 03:35:22 [DEBUG   ] summary = ERROR
2015-11-10 03:35:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533010  13 57336 12922675 A  ERRORERROR'
2015-11-10 03:35:22 [DEBUG   ] Got command RPT from MCS: ref #533011
2015-11-10 03:35:22 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:35:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533011  77 57336 12922879 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:35:23 [DEBUG   ] Got command RPT from MCS: ref #533012
2015-11-10 03:35:23 [DEBUG   ] summary = ERROR
2015-11-10 03:35:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533012  13 57336 12923083 A  ERRORERROR'
2015-11-10 03:35:23 [DEBUG   ] Got command RPT from MCS: ref #533013
2015-11-10 03:35:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:35:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533013  77 57336 12923287 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:35:23 [DEBUG   ] Got command RPT from MCS: ref #533014
2015-11-10 03:35:23 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 03:35:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533014  12 57336 12923479 A  ERRORB�™š'
2015-11-10 03:35:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:35:46 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:35:48 [DEBUG   ] Finished updating board flags in 1.845 seconds
2015-11-10 03:35:58 [DEBUG   ] Got command DRX from MCS: ref #533023
2015-11-10 03:35:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:35:58 [DEBUG   ] run_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:35:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533023   8 57336 12958007 A  ERROR'
2015-11-10 03:35:58 [INFO    ] Finished starting DRX
2015-11-10 03:35:58 [DEBUG   ] Got command DRX from MCS: ref #533024
2015-11-10 03:35:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:35:58 [DEBUG   ] run_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:35:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533024   8 57336 12958179 A  ERROR'
2015-11-10 03:35:58 [DEBUG   ] Got command BAM from MCS: ref #533025
2015-11-10 03:35:58 [INFO    ] Finished processing BAM request
2015-11-10 03:35:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533025   8 57336 12958183 A  ERROR'
2015-11-10 03:35:58 [INFO    ] Finished starting DRX
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 03:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 03:35:58 [DEBUG   ] spawn_process: at 2015-11-10 03:35:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 03:36:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:36:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:36:46 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:36:47 [DEBUG   ] Finished updating board flags in 1.223 seconds
2015-11-10 03:37:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:37:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:37:46 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:37:48 [DEBUG   ] Finished updating board flags in 1.766 seconds
2015-11-10 03:37:48 [DEBUG   ] Got command RPT from MCS: ref #533026
2015-11-10 03:37:48 [DEBUG   ] summary = ERROR
2015-11-10 03:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533026  13 57336 13068991 A  ERRORERROR'
2015-11-10 03:37:48 [DEBUG   ] Got command RPT from MCS: ref #533027
2015-11-10 03:37:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533027  77 57336 13068991 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:37:58 [DEBUG   ] Got command DRX from MCS: ref #533043
2015-11-10 03:37:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:37:58 [DEBUG   ] run_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:37:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533043   8 57336 13078007 A  ERROR'
2015-11-10 03:37:58 [INFO    ] Finished starting DRX
2015-11-10 03:37:58 [DEBUG   ] Got command DRX from MCS: ref #533044
2015-11-10 03:37:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:37:58 [DEBUG   ] run_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:37:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533044   8 57336 13078179 A  ERROR'
2015-11-10 03:37:58 [DEBUG   ] Got command BAM from MCS: ref #533045
2015-11-10 03:37:58 [INFO    ] Finished processing BAM request
2015-11-10 03:37:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533045   8 57336 13078183 A  ERROR'
2015-11-10 03:37:58 [INFO    ] Finished starting DRX
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 03:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 03:37:58 [DEBUG   ] spawn_process: at 2015-11-10 03:37:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 03:38:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:38:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:38:46 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:38:48 [DEBUG   ] Finished updating board flags in 1.580 seconds
2015-11-10 03:39:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:39:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:39:46 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:39:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 03:39:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 03:39:48 [DEBUG   ] Finished updating board flags in 1.423 seconds
2015-11-10 03:39:58 [DEBUG   ] Got command DRX from MCS: ref #533049
2015-11-10 03:39:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:39:58 [DEBUG   ] run_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533049   8 57336 13198011 A  ERROR'
2015-11-10 03:39:58 [DEBUG   ] Got command DRX from MCS: ref #533050
2015-11-10 03:39:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:39:58 [DEBUG   ] run_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533050   8 57336 13198015 A  ERROR'
2015-11-10 03:39:58 [INFO    ] Finished starting DRX
2015-11-10 03:39:58 [INFO    ] Finished starting DRX
2015-11-10 03:39:58 [DEBUG   ] Got command BAM from MCS: ref #533051
2015-11-10 03:39:58 [INFO    ] Finished processing BAM request
2015-11-10 03:39:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533051   8 57336 13198187 A  ERROR'
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 03:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 03:39:58 [DEBUG   ] spawn_process: at 2015-11-10 03:39:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 03:40:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 03:40:22 [DEBUG   ] Got command RPT from MCS: ref #533052
2015-11-10 03:40:22 [DEBUG   ] summary = ERROR
2015-11-10 03:40:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533052  13 57336 13222771 A  ERRORERROR'
2015-11-10 03:40:22 [DEBUG   ] Got command RPT from MCS: ref #533053
2015-11-10 03:40:22 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:40:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533053  77 57336 13222975 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:40:23 [DEBUG   ] Got command RPT from MCS: ref #533054
2015-11-10 03:40:23 [DEBUG   ] summary = ERROR
2015-11-10 03:40:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533054  13 57336 13223179 A  ERRORERROR'
2015-11-10 03:40:23 [DEBUG   ] Got command RPT from MCS: ref #533055
2015-11-10 03:40:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:40:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533055  77 57336 13223383 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:40:23 [DEBUG   ] Got command RPT from MCS: ref #533056
2015-11-10 03:40:23 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 03:40:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533056  12 57336 13223575 A  ERRORB�™š'
2015-11-10 03:40:26 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-10 03:40:46 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 03:40:48 [DEBUG   ] Finished updating board flags in 1.371 seconds
2015-11-10 03:41:06 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 03:41:26 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 03:41:46 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 03:41:48 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 03:41:48 [DEBUG   ] Finished updating board flags in 1.221 seconds
2015-11-10 03:41:58 [DEBUG   ] Got command DRX from MCS: ref #533060
2015-11-10 03:41:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:41:58 [DEBUG   ] run_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:41:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533060   8 57336 13318011 A  ERROR'
2015-11-10 03:41:58 [DEBUG   ] Got command DRX from MCS: ref #533061
2015-11-10 03:41:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:41:58 [DEBUG   ] run_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:41:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533061   8 57336 13318015 A  ERROR'
2015-11-10 03:41:58 [INFO    ] Finished starting DRX
2015-11-10 03:41:58 [INFO    ] Finished starting DRX
2015-11-10 03:41:58 [DEBUG   ] Got command BAM from MCS: ref #533062
2015-11-10 03:41:58 [INFO    ] Finished processing BAM request
2015-11-10 03:41:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533062   8 57336 13318235 A  ERROR'
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 03:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 03:41:58 [DEBUG   ] spawn_process: at 2015-11-10 03:41:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 03:42:06 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 03:42:26 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 03:42:46 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 03:42:48 [DEBUG   ] Finished updating board flags in 1.184 seconds
2015-11-10 03:42:48 [DEBUG   ] Got command RPT from MCS: ref #533063
2015-11-10 03:42:48 [DEBUG   ] summary = ERROR
2015-11-10 03:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533063  13 57336 13368991 A  ERRORERROR'
2015-11-10 03:42:48 [DEBUG   ] Got command RPT from MCS: ref #533064
2015-11-10 03:42:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533064  77 57336 13368991 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:43:06 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 03:43:26 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 03:43:46 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 03:43:48 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 03:43:48 [DEBUG   ] Finished updating board flags in 1.039 seconds
2015-11-10 03:43:58 [DEBUG   ] Got command DRX from MCS: ref #533080
2015-11-10 03:43:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:43:58 [DEBUG   ] run_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:43:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533080   8 57336 13438015 A  ERROR'
2015-11-10 03:43:58 [DEBUG   ] Got command DRX from MCS: ref #533081
2015-11-10 03:43:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:43:58 [DEBUG   ] run_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:43:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533081   8 57336 13438019 A  ERROR'
2015-11-10 03:43:58 [INFO    ] Finished starting DRX
2015-11-10 03:43:58 [INFO    ] Finished starting DRX
2015-11-10 03:43:58 [DEBUG   ] Got command BAM from MCS: ref #533082
2015-11-10 03:43:58 [INFO    ] Finished processing BAM request
2015-11-10 03:43:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533082   8 57336 13438167 A  ERROR'
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 03:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 03:43:58 [DEBUG   ] spawn_process: at 2015-11-10 03:43:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 03:44:06 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-10 03:44:26 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:44:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:44:49 [DEBUG   ] Finished updating board flags in 1.996 seconds
2015-11-10 03:45:06 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:45:22 [DEBUG   ] Got command RPT from MCS: ref #533087
2015-11-10 03:45:22 [DEBUG   ] summary = ERROR
2015-11-10 03:45:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533087  13 57336 13522867 A  ERRORERROR'
2015-11-10 03:45:23 [DEBUG   ] Got command RPT from MCS: ref #533088
2015-11-10 03:45:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:45:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533088  77 57336 13523071 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:45:23 [DEBUG   ] Got command RPT from MCS: ref #533089
2015-11-10 03:45:23 [DEBUG   ] summary = ERROR
2015-11-10 03:45:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533089  13 57336 13523275 A  ERRORERROR'
2015-11-10 03:45:23 [DEBUG   ] Got command RPT from MCS: ref #533090
2015-11-10 03:45:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:45:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533090  77 57336 13523479 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:45:23 [DEBUG   ] Got command RPT from MCS: ref #533091
2015-11-10 03:45:23 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 03:45:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533091  12 57336 13523671 A  ERRORB�™š'
2015-11-10 03:45:26 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:45:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:45:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 03:45:48 [DEBUG   ] Finished updating board flags in 1.256 seconds
2015-11-10 03:45:58 [DEBUG   ] Got command DRX from MCS: ref #533097
2015-11-10 03:45:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:45:58 [DEBUG   ] run_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:45:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533097   8 57336 13558015 A  ERROR'
2015-11-10 03:45:58 [DEBUG   ] Got command DRX from MCS: ref #533098
2015-11-10 03:45:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:45:58 [DEBUG   ] run_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:45:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533098   8 57336 13558019 A  ERROR'
2015-11-10 03:45:58 [INFO    ] Finished starting DRX
2015-11-10 03:45:58 [INFO    ] Finished starting DRX
2015-11-10 03:45:58 [DEBUG   ] Got command BAM from MCS: ref #533099
2015-11-10 03:45:58 [INFO    ] Finished processing BAM request
2015-11-10 03:45:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533099   8 57336 13558179 A  ERROR'
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 03:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 03:45:58 [DEBUG   ] spawn_process: at 2015-11-10 03:45:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 03:46:06 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:46:26 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:46:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:46:48 [DEBUG   ] Finished updating board flags in 1.236 seconds
2015-11-10 03:47:06 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 03:47:26 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2015-11-10 03:47:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:47:48 [DEBUG   ] Got command RPT from MCS: ref #533103
2015-11-10 03:47:48 [DEBUG   ] summary = ERROR
2015-11-10 03:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533103  13 57336 13668987 A  ERRORERROR'
2015-11-10 03:47:48 [DEBUG   ] Got command RPT from MCS: ref #533104
2015-11-10 03:47:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533104  77 57336 13668987 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:47:49 [DEBUG   ] Finished updating board flags in 1.741 seconds
2015-11-10 03:47:58 [DEBUG   ] Got command DRX from MCS: ref #533120
2015-11-10 03:47:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:47:58 [DEBUG   ] run_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:47:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533120   8 57336 13678015 A  ERROR'
2015-11-10 03:47:58 [DEBUG   ] Got command DRX from MCS: ref #533121
2015-11-10 03:47:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:47:58 [DEBUG   ] run_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:47:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533121   8 57336 13678019 A  ERROR'
2015-11-10 03:47:58 [INFO    ] Finished starting DRX
2015-11-10 03:47:58 [INFO    ] Finished starting DRX
2015-11-10 03:47:58 [DEBUG   ] Got command BAM from MCS: ref #533122
2015-11-10 03:47:58 [INFO    ] Finished processing BAM request
2015-11-10 03:47:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533122   8 57336 13678203 A  ERROR'
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 03:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 03:47:58 [DEBUG   ] spawn_process: at 2015-11-10 03:47:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 03:48:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:48:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:48:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:48:48 [DEBUG   ] Finished updating board flags in 0.983 seconds
2015-11-10 03:49:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:49:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:49:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:49:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 03:49:49 [DEBUG   ] Finished updating board flags in 1.371 seconds
2015-11-10 03:49:58 [DEBUG   ] Got command DRX from MCS: ref #533123
2015-11-10 03:49:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:49:58 [DEBUG   ] run_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:49:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533123   8 57336 13798007 A  ERROR'
2015-11-10 03:49:58 [INFO    ] Finished starting DRX
2015-11-10 03:49:58 [DEBUG   ] Got command DRX from MCS: ref #533124
2015-11-10 03:49:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:49:58 [DEBUG   ] run_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:49:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533124   8 57336 13798203 A  ERROR'
2015-11-10 03:49:58 [DEBUG   ] Got command BAM from MCS: ref #533125
2015-11-10 03:49:58 [INFO    ] Finished processing BAM request
2015-11-10 03:49:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533125   8 57336 13798207 A  ERROR'
2015-11-10 03:49:58 [INFO    ] Finished starting DRX
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 03:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 03:49:58 [DEBUG   ] spawn_process: at 2015-11-10 03:49:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 03:50:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:50:22 [DEBUG   ] Got command RPT from MCS: ref #533126
2015-11-10 03:50:22 [DEBUG   ] summary = ERROR
2015-11-10 03:50:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533126  13 57336 13822963 A  ERRORERROR'
2015-11-10 03:50:23 [DEBUG   ] Got command RPT from MCS: ref #533127
2015-11-10 03:50:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:50:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533127  77 57336 13823167 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:50:23 [DEBUG   ] Got command RPT from MCS: ref #533128
2015-11-10 03:50:23 [DEBUG   ] summary = ERROR
2015-11-10 03:50:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533128  13 57336 13823371 A  ERRORERROR'
2015-11-10 03:50:23 [DEBUG   ] Got command RPT from MCS: ref #533129
2015-11-10 03:50:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:50:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533129  77 57336 13823563 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:50:23 [DEBUG   ] Got command RPT from MCS: ref #533130
2015-11-10 03:50:23 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 03:50:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533130  12 57336 13823767 A  ERRORB�ÌÍ'
2015-11-10 03:50:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:50:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:50:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 03:50:49 [DEBUG   ] Finished updating board flags in 1.214 seconds
2015-11-10 03:51:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:51:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:51:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:51:49 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 03:51:49 [DEBUG   ] Finished updating board flags in 1.182 seconds
2015-11-10 03:51:58 [DEBUG   ] Got command DRX from MCS: ref #533134
2015-11-10 03:51:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:51:58 [DEBUG   ] run_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:51:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533134   8 57336 13918011 A  ERROR'
2015-11-10 03:51:58 [DEBUG   ] Got command DRX from MCS: ref #533135
2015-11-10 03:51:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:51:58 [DEBUG   ] run_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:51:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533135   8 57336 13918015 A  ERROR'
2015-11-10 03:51:58 [INFO    ] Finished starting DRX
2015-11-10 03:51:58 [INFO    ] Finished starting DRX
2015-11-10 03:51:58 [DEBUG   ] Got command BAM from MCS: ref #533136
2015-11-10 03:51:58 [INFO    ] Finished processing BAM request
2015-11-10 03:51:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533136   8 57336 13918199 A  ERROR'
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 03:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 03:51:58 [DEBUG   ] spawn_process: at 2015-11-10 03:51:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 03:52:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:52:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:52:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:52:48 [DEBUG   ] Got command RPT from MCS: ref #533137
2015-11-10 03:52:48 [DEBUG   ] summary = ERROR
2015-11-10 03:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533137  13 57336 13968995 A  ERRORERROR'
2015-11-10 03:52:48 [DEBUG   ] Got command RPT from MCS: ref #533138
2015-11-10 03:52:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533138  77 57336 13968995 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:52:49 [DEBUG   ] Finished updating board flags in 1.040 seconds
2015-11-10 03:53:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:53:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:53:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:53:50 [DEBUG   ] Finished updating board flags in 2.041 seconds
2015-11-10 03:53:58 [DEBUG   ] Got command DRX from MCS: ref #533157
2015-11-10 03:53:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:53:58 [DEBUG   ] run_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:53:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533157   8 57336 14038007 A  ERROR'
2015-11-10 03:53:58 [INFO    ] Finished starting DRX
2015-11-10 03:53:58 [DEBUG   ] Got command DRX from MCS: ref #533158
2015-11-10 03:53:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:53:58 [DEBUG   ] run_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:53:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533158   8 57336 14038191 A  ERROR'
2015-11-10 03:53:58 [DEBUG   ] Got command BAM from MCS: ref #533159
2015-11-10 03:53:58 [INFO    ] Finished processing BAM request
2015-11-10 03:53:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533159   8 57336 14038195 A  ERROR'
2015-11-10 03:53:58 [INFO    ] Finished starting DRX
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 03:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 03:53:58 [DEBUG   ] spawn_process: at 2015-11-10 03:53:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 03:54:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:54:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:54:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:54:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 03:54:50 [DEBUG   ] Finished updating board flags in 1.918 seconds
2015-11-10 03:55:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:55:23 [DEBUG   ] Got command RPT from MCS: ref #533161
2015-11-10 03:55:23 [DEBUG   ] summary = ERROR
2015-11-10 03:55:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533161  13 57336 14123059 A  ERRORERROR'
2015-11-10 03:55:23 [DEBUG   ] Got command RPT from MCS: ref #533162
2015-11-10 03:55:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:55:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533162  77 57336 14123263 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:55:23 [DEBUG   ] Got command RPT from MCS: ref #533163
2015-11-10 03:55:23 [DEBUG   ] summary = ERROR
2015-11-10 03:55:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533163  13 57336 14123467 A  ERRORERROR'
2015-11-10 03:55:23 [DEBUG   ] Got command RPT from MCS: ref #533164
2015-11-10 03:55:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:55:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533164  77 57336 14123659 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:55:23 [DEBUG   ] Got command RPT from MCS: ref #533165
2015-11-10 03:55:23 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 03:55:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533165  12 57336 14123863 A  ERRORB�™š'
2015-11-10 03:55:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:55:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:55:49 [DEBUG   ] Finished updating board flags in 1.267 seconds
2015-11-10 03:55:58 [DEBUG   ] Got command DRX from MCS: ref #533171
2015-11-10 03:55:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:55:58 [DEBUG   ] run_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:55:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533171   8 57336 14158015 A  ERROR'
2015-11-10 03:55:58 [DEBUG   ] Got command DRX from MCS: ref #533172
2015-11-10 03:55:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:55:58 [DEBUG   ] run_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:55:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533172   8 57336 14158019 A  ERROR'
2015-11-10 03:55:58 [INFO    ] Finished starting DRX
2015-11-10 03:55:58 [INFO    ] Finished starting DRX
2015-11-10 03:55:58 [DEBUG   ] Got command BAM from MCS: ref #533173
2015-11-10 03:55:58 [INFO    ] Finished processing BAM request
2015-11-10 03:55:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533173   8 57336 14158167 A  ERROR'
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 03:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 03:55:58 [DEBUG   ] spawn_process: at 2015-11-10 03:55:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 03:56:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 03:56:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:56:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:56:49 [DEBUG   ] Finished updating board flags in 1.224 seconds
2015-11-10 03:57:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 03:57:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:57:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:57:48 [DEBUG   ] Got command RPT from MCS: ref #533177
2015-11-10 03:57:48 [DEBUG   ] summary = ERROR
2015-11-10 03:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533177  13 57336 14268991 A  ERRORERROR'
2015-11-10 03:57:48 [DEBUG   ] Got command RPT from MCS: ref #533178
2015-11-10 03:57:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 03:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533178  77 57336 14268991 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 03:57:49 [DEBUG   ] Finished updating board flags in 1.186 seconds
2015-11-10 03:57:58 [DEBUG   ] Got command DRX from MCS: ref #533194
2015-11-10 03:57:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:57:58 [DEBUG   ] run_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:57:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533194   8 57336 14278007 A  ERROR'
2015-11-10 03:57:58 [INFO    ] Finished starting DRX
2015-11-10 03:57:58 [DEBUG   ] Got command DRX from MCS: ref #533195
2015-11-10 03:57:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:57:58 [DEBUG   ] run_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:57:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533195   8 57336 14278191 A  ERROR'
2015-11-10 03:57:58 [DEBUG   ] Got command BAM from MCS: ref #533196
2015-11-10 03:57:58 [INFO    ] Finished processing BAM request
2015-11-10 03:57:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533196   8 57336 14278195 A  ERROR'
2015-11-10 03:57:58 [INFO    ] Finished starting DRX
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 03:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 03:57:58 [DEBUG   ] spawn_process: at 2015-11-10 03:57:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 03:58:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:58:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:58:46 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 03:58:50 [DEBUG   ] Finished updating board flags in 1.588 seconds
2015-11-10 03:59:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:59:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:59:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 03:59:50 [DEBUG   ] Finished updating board flags in 1.433 seconds
2015-11-10 03:59:58 [DEBUG   ] Got command DRX from MCS: ref #533200
2015-11-10 03:59:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:59:58 [DEBUG   ] run_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 03:59:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533200   8 57336 14398011 A  ERROR'
2015-11-10 03:59:58 [DEBUG   ] Got command DRX from MCS: ref #533201
2015-11-10 03:59:58 [INFO    ] sending drx command to lwa16
2015-11-10 03:59:58 [DEBUG   ] run_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 03:59:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 03:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533201   8 57336 14398015 A  ERROR'
2015-11-10 03:59:58 [INFO    ] Finished starting DRX
2015-11-10 03:59:58 [INFO    ] Finished starting DRX
2015-11-10 03:59:58 [DEBUG   ] Got command BAM from MCS: ref #533202
2015-11-10 03:59:58 [INFO    ] Finished processing BAM request
2015-11-10 03:59:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 03:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533202   8 57336 14398187 A  ERROR'
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 03:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 03:59:58 [DEBUG   ] spawn_process: at 2015-11-10 03:59:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 04:00:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:00:23 [DEBUG   ] Got command RPT from MCS: ref #533203
2015-11-10 04:00:23 [DEBUG   ] summary = ERROR
2015-11-10 04:00:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533203  13 57336 14423139 A  ERRORERROR'
2015-11-10 04:00:23 [DEBUG   ] Got command RPT from MCS: ref #533204
2015-11-10 04:00:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:00:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533204  77 57336 14423343 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:00:23 [DEBUG   ] Got command RPT from MCS: ref #533205
2015-11-10 04:00:23 [DEBUG   ] summary = ERROR
2015-11-10 04:00:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533205  13 57336 14423547 A  ERRORERROR'
2015-11-10 04:00:23 [DEBUG   ] Got command RPT from MCS: ref #533206
2015-11-10 04:00:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:00:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533206  77 57336 14423751 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:00:23 [DEBUG   ] Got command RPT from MCS: ref #533207
2015-11-10 04:00:23 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 04:00:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533207  12 57336 14423943 A  ERRORB‘™š'
2015-11-10 04:00:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:00:46 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:00:50 [DEBUG   ] Finished updating board flags in 1.317 seconds
2015-11-10 04:01:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:01:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:01:46 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:01:50 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:01:50 [DEBUG   ] Finished updating board flags in 1.217 seconds
2015-11-10 04:01:58 [DEBUG   ] Got command DRX from MCS: ref #533208
2015-11-10 04:01:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:01:58 [DEBUG   ] run_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:01:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533208   8 57336 14518015 A  ERROR'
2015-11-10 04:01:58 [DEBUG   ] Got command DRX from MCS: ref #533209
2015-11-10 04:01:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:01:58 [DEBUG   ] run_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:01:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533209   8 57336 14518019 A  ERROR'
2015-11-10 04:01:58 [INFO    ] Finished starting DRX
2015-11-10 04:01:58 [INFO    ] Finished starting DRX
2015-11-10 04:01:58 [DEBUG   ] Got command BAM from MCS: ref #533210
2015-11-10 04:01:58 [INFO    ] Finished processing BAM request
2015-11-10 04:01:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533210   8 57336 14518155 A  ERROR'
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 04:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 04:01:58 [DEBUG   ] spawn_process: at 2015-11-10 04:01:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 04:02:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:02:26 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:02:46 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:02:49 [DEBUG   ] Got command RPT from MCS: ref #533214
2015-11-10 04:02:49 [DEBUG   ] summary = ERROR
2015-11-10 04:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533214  13 57336 14569007 A  ERRORERROR'
2015-11-10 04:02:49 [DEBUG   ] Got command RPT from MCS: ref #533215
2015-11-10 04:02:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533215  77 57336 14569007 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:02:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 04:02:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:02:50 [DEBUG   ] Finished updating board flags in 1.183 seconds
2015-11-10 04:03:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:03:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:03:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:03:50 [DEBUG   ] Finished updating board flags in 1.016 seconds
2015-11-10 04:03:58 [DEBUG   ] Got command DRX from MCS: ref #533231
2015-11-10 04:03:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:03:58 [DEBUG   ] run_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:03:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533231   8 57336 14638007 A  ERROR'
2015-11-10 04:03:58 [INFO    ] Finished starting DRX
2015-11-10 04:03:58 [DEBUG   ] Got command DRX from MCS: ref #533232
2015-11-10 04:03:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:03:58 [DEBUG   ] run_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:03:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533232   8 57336 14638179 A  ERROR'
2015-11-10 04:03:58 [DEBUG   ] Got command BAM from MCS: ref #533233
2015-11-10 04:03:58 [INFO    ] Finished processing BAM request
2015-11-10 04:03:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533233   8 57336 14638183 A  ERROR'
2015-11-10 04:03:58 [INFO    ] Finished starting DRX
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 04:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 04:03:58 [DEBUG   ] spawn_process: at 2015-11-10 04:03:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 04:04:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:04:26 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:04:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:04:50 [DEBUG   ] Finished updating board flags in 1.057 seconds
2015-11-10 04:05:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:05:23 [DEBUG   ] Got command RPT from MCS: ref #533235
2015-11-10 04:05:23 [DEBUG   ] summary = ERROR
2015-11-10 04:05:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533235  13 57336 14723231 A  ERRORERROR'
2015-11-10 04:05:23 [DEBUG   ] Got command RPT from MCS: ref #533236
2015-11-10 04:05:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:05:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533236  77 57336 14723435 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:05:23 [DEBUG   ] Got command RPT from MCS: ref #533237
2015-11-10 04:05:23 [DEBUG   ] summary = ERROR
2015-11-10 04:05:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533237  13 57336 14723639 A  ERRORERROR'
2015-11-10 04:05:23 [DEBUG   ] Got command RPT from MCS: ref #533238
2015-11-10 04:05:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:05:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533238  77 57336 14723831 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:05:24 [DEBUG   ] Got command RPT from MCS: ref #533239
2015-11-10 04:05:24 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 04:05:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533239  12 57336 14724035 A  ERRORB�ÌÍ'
2015-11-10 04:05:26 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:05:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:05:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 04:05:51 [DEBUG   ] Finished updating board flags in 2.014 seconds
2015-11-10 04:05:58 [DEBUG   ] Got command DRX from MCS: ref #533248
2015-11-10 04:05:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:05:58 [DEBUG   ] run_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:05:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533248   8 57336 14758015 A  ERROR'
2015-11-10 04:05:58 [DEBUG   ] Got command DRX from MCS: ref #533249
2015-11-10 04:05:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:05:58 [DEBUG   ] run_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:05:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533249   8 57336 14758019 A  ERROR'
2015-11-10 04:05:58 [INFO    ] Finished starting DRX
2015-11-10 04:05:58 [INFO    ] Finished starting DRX
2015-11-10 04:05:58 [DEBUG   ] Got command BAM from MCS: ref #533250
2015-11-10 04:05:58 [INFO    ] Finished processing BAM request
2015-11-10 04:05:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533250   8 57336 14758251 A  ERROR'
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 04:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 04:05:58 [DEBUG   ] spawn_process: at 2015-11-10 04:05:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 04:06:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:06:26 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:06:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:06:50 [DEBUG   ] Finished updating board flags in 1.374 seconds
2015-11-10 04:07:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:07:26 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:07:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:07:49 [DEBUG   ] Got command RPT from MCS: ref #533251
2015-11-10 04:07:49 [DEBUG   ] summary = ERROR
2015-11-10 04:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533251  13 57336 14869007 A  ERRORERROR'
2015-11-10 04:07:49 [DEBUG   ] Got command RPT from MCS: ref #533252
2015-11-10 04:07:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533252  77 57336 14869007 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:07:50 [DEBUG   ] Finished updating board flags in 1.221 seconds
2015-11-10 04:07:58 [DEBUG   ] Got command DRX from MCS: ref #533268
2015-11-10 04:07:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:07:58 [DEBUG   ] run_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:07:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533268   8 57336 14878007 A  ERROR'
2015-11-10 04:07:58 [INFO    ] Finished starting DRX
2015-11-10 04:07:58 [DEBUG   ] Got command DRX from MCS: ref #533269
2015-11-10 04:07:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:07:58 [DEBUG   ] run_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:07:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533269   8 57336 14878239 A  ERROR'
2015-11-10 04:07:58 [DEBUG   ] Got command BAM from MCS: ref #533270
2015-11-10 04:07:58 [INFO    ] Finished processing BAM request
2015-11-10 04:07:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533270   8 57336 14878243 A  ERROR'
2015-11-10 04:07:58 [INFO    ] Finished starting DRX
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 04:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 04:07:58 [DEBUG   ] spawn_process: at 2015-11-10 04:07:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 04:08:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:08:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:08:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:08:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:08:50 [DEBUG   ] Finished updating board flags in 1.211 seconds
2015-11-10 04:09:06 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 04:09:26 [DEBUG   ] Finished updating ant stats in 0.332 seconds
2015-11-10 04:09:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:09:50 [DEBUG   ] Finished updating board flags in 1.163 seconds
2015-11-10 04:09:58 [DEBUG   ] Got command DRX from MCS: ref #533274
2015-11-10 04:09:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:09:58 [DEBUG   ] run_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:09:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533274   8 57336 14998007 A  ERROR'
2015-11-10 04:09:58 [INFO    ] Finished starting DRX
2015-11-10 04:09:58 [DEBUG   ] Got command DRX from MCS: ref #533275
2015-11-10 04:09:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:09:58 [DEBUG   ] run_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:09:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533275   8 57336 14998179 A  ERROR'
2015-11-10 04:09:58 [DEBUG   ] Got command BAM from MCS: ref #533276
2015-11-10 04:09:58 [INFO    ] Finished processing BAM request
2015-11-10 04:09:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533276   8 57336 14998183 A  ERROR'
2015-11-10 04:09:58 [INFO    ] Finished starting DRX
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 04:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 04:09:58 [DEBUG   ] spawn_process: at 2015-11-10 04:09:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 04:10:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:10:23 [DEBUG   ] Got command RPT from MCS: ref #533277
2015-11-10 04:10:23 [DEBUG   ] summary = ERROR
2015-11-10 04:10:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533277  13 57336 15023323 A  ERRORERROR'
2015-11-10 04:10:23 [DEBUG   ] Got command RPT from MCS: ref #533278
2015-11-10 04:10:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:10:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533278  77 57336 15023731 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:10:23 [DEBUG   ] Got command RPT from MCS: ref #533279
2015-11-10 04:10:23 [DEBUG   ] summary = ERROR
2015-11-10 04:10:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533279  13 57336 15023923 A  ERRORERROR'
2015-11-10 04:10:24 [DEBUG   ] Got command RPT from MCS: ref #533280
2015-11-10 04:10:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:10:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533280  77 57336 15024127 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:10:24 [DEBUG   ] Got command RPT from MCS: ref #533281
2015-11-10 04:10:24 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 04:10:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533281  12 57336 15024331 A  ERRORB�™š'
2015-11-10 04:10:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:10:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:10:50 [DEBUG   ] Finished updating board flags in 1.023 seconds
2015-11-10 04:11:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:11:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:11:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:11:50 [DEBUG   ] Finished updating board flags in 0.998 seconds
2015-11-10 04:11:58 [DEBUG   ] Got command DRX from MCS: ref #533285
2015-11-10 04:11:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:11:58 [DEBUG   ] run_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:11:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533285   8 57336 15118011 A  ERROR'
2015-11-10 04:11:58 [DEBUG   ] Got command DRX from MCS: ref #533286
2015-11-10 04:11:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:11:58 [DEBUG   ] run_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:11:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533286   8 57336 15118015 A  ERROR'
2015-11-10 04:11:58 [INFO    ] Finished starting DRX
2015-11-10 04:11:58 [INFO    ] Finished starting DRX
2015-11-10 04:11:58 [DEBUG   ] Got command BAM from MCS: ref #533287
2015-11-10 04:11:58 [INFO    ] Finished processing BAM request
2015-11-10 04:11:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533287   8 57336 15118163 A  ERROR'
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 04:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 04:11:58 [DEBUG   ] spawn_process: at 2015-11-10 04:11:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 04:12:07 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 04:12:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:12:47 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 04:12:49 [DEBUG   ] Got command RPT from MCS: ref #533288
2015-11-10 04:12:49 [DEBUG   ] summary = ERROR
2015-11-10 04:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533288  13 57336 15169011 A  ERRORERROR'
2015-11-10 04:12:49 [DEBUG   ] Got command RPT from MCS: ref #533289
2015-11-10 04:12:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533289  77 57336 15169011 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:12:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 04:12:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:12:51 [DEBUG   ] Finished updating board flags in 1.404 seconds
2015-11-10 04:13:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:13:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:13:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:13:51 [DEBUG   ] Finished updating board flags in 1.363 seconds
2015-11-10 04:13:58 [DEBUG   ] Got command DRX from MCS: ref #533305
2015-11-10 04:13:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:13:58 [DEBUG   ] run_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:13:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533305   8 57336 15238011 A  ERROR'
2015-11-10 04:13:58 [DEBUG   ] Got command DRX from MCS: ref #533306
2015-11-10 04:13:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:13:58 [DEBUG   ] run_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:13:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533306   8 57336 15238015 A  ERROR'
2015-11-10 04:13:58 [INFO    ] Finished starting DRX
2015-11-10 04:13:58 [INFO    ] Finished starting DRX
2015-11-10 04:13:58 [DEBUG   ] Got command BAM from MCS: ref #533307
2015-11-10 04:13:58 [INFO    ] Finished processing BAM request
2015-11-10 04:13:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533307   8 57336 15238175 A  ERROR'
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 04:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 04:13:58 [DEBUG   ] spawn_process: at 2015-11-10 04:13:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 04:14:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:14:27 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 04:14:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:14:51 [DEBUG   ] Finished updating board flags in 1.183 seconds
2015-11-10 04:15:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:15:23 [DEBUG   ] Got command RPT from MCS: ref #533312
2015-11-10 04:15:23 [DEBUG   ] summary = ERROR
2015-11-10 04:15:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533312  13 57336 15323447 A  ERRORERROR'
2015-11-10 04:15:23 [DEBUG   ] Got command RPT from MCS: ref #533313
2015-11-10 04:15:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:15:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533313  77 57336 15323639 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:15:23 [DEBUG   ] Got command RPT from MCS: ref #533314
2015-11-10 04:15:23 [DEBUG   ] summary = ERROR
2015-11-10 04:15:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533314  13 57336 15323843 A  ERRORERROR'
2015-11-10 04:15:24 [DEBUG   ] Got command RPT from MCS: ref #533315
2015-11-10 04:15:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:15:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533315  77 57336 15324047 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:15:24 [DEBUG   ] Got command RPT from MCS: ref #533316
2015-11-10 04:15:24 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 04:15:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533316  12 57336 15324239 A  ERRORB�™š'
2015-11-10 04:15:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 04:15:47 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 04:15:51 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:15:51 [DEBUG   ] Finished updating board flags in 1.015 seconds
2015-11-10 04:15:58 [DEBUG   ] Got command DRX from MCS: ref #533322
2015-11-10 04:15:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:15:58 [DEBUG   ] run_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:15:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533322   8 57336 15358011 A  ERROR'
2015-11-10 04:15:58 [DEBUG   ] Got command DRX from MCS: ref #533323
2015-11-10 04:15:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:15:58 [DEBUG   ] run_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:15:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533323   8 57336 15358015 A  ERROR'
2015-11-10 04:15:58 [INFO    ] Finished starting DRX
2015-11-10 04:15:58 [INFO    ] Finished starting DRX
2015-11-10 04:15:58 [DEBUG   ] Got command BAM from MCS: ref #533324
2015-11-10 04:15:58 [INFO    ] Finished processing BAM request
2015-11-10 04:15:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533324   8 57336 15358151 A  ERROR'
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 04:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 04:15:58 [DEBUG   ] spawn_process: at 2015-11-10 04:15:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 04:16:07 [DEBUG   ] Finished updating ant stats in 1.192 seconds
2015-11-10 04:16:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 04:16:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 04:16:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 04:16:52 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:16:52 [DEBUG   ] Finished updating board flags in 1.995 seconds
2015-11-10 04:17:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 04:17:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 04:17:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 04:17:49 [DEBUG   ] Got command RPT from MCS: ref #533328
2015-11-10 04:17:49 [DEBUG   ] summary = ERROR
2015-11-10 04:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533328  13 57336 15469011 A  ERRORERROR'
2015-11-10 04:17:49 [DEBUG   ] Got command RPT from MCS: ref #533329
2015-11-10 04:17:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533329  77 57336 15469011 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:17:51 [DEBUG   ] Finished updating board flags in 1.236 seconds
2015-11-10 04:17:58 [DEBUG   ] Got command DRX from MCS: ref #533345
2015-11-10 04:17:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:17:58 [DEBUG   ] run_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:17:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533345   8 57336 15478007 A  ERROR'
2015-11-10 04:17:58 [INFO    ] Finished starting DRX
2015-11-10 04:17:58 [DEBUG   ] Got command DRX from MCS: ref #533346
2015-11-10 04:17:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:17:58 [DEBUG   ] run_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:17:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533346   8 57336 15478167 A  ERROR'
2015-11-10 04:17:58 [DEBUG   ] Got command BAM from MCS: ref #533347
2015-11-10 04:17:58 [INFO    ] Finished processing BAM request
2015-11-10 04:17:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533347   8 57336 15478171 A  ERROR'
2015-11-10 04:17:58 [INFO    ] Finished starting DRX
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 04:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 04:17:58 [DEBUG   ] spawn_process: at 2015-11-10 04:17:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 04:18:07 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2015-11-10 04:18:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 04:18:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 04:18:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:18:52 [DEBUG   ] Finished updating board flags in 1.752 seconds
2015-11-10 04:19:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 04:19:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 04:19:47 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 04:19:51 [DEBUG   ] Finished updating board flags in 1.060 seconds
2015-11-10 04:19:58 [DEBUG   ] Got command DRX from MCS: ref #533348
2015-11-10 04:19:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:19:58 [DEBUG   ] run_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:19:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533348   8 57336 15598011 A  ERROR'
2015-11-10 04:19:58 [DEBUG   ] Got command DRX from MCS: ref #533349
2015-11-10 04:19:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:19:58 [DEBUG   ] run_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:19:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533349   8 57336 15598015 A  ERROR'
2015-11-10 04:19:58 [INFO    ] Finished starting DRX
2015-11-10 04:19:58 [INFO    ] Finished starting DRX
2015-11-10 04:19:58 [DEBUG   ] Got command BAM from MCS: ref #533350
2015-11-10 04:19:58 [INFO    ] Finished processing BAM request
2015-11-10 04:19:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533350   8 57336 15598163 A  ERROR'
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 04:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 04:19:58 [DEBUG   ] spawn_process: at 2015-11-10 04:19:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 04:20:07 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 04:20:23 [DEBUG   ] Got command RPT from MCS: ref #533351
2015-11-10 04:20:23 [DEBUG   ] summary = ERROR
2015-11-10 04:20:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533351  13 57336 15623535 A  ERRORERROR'
2015-11-10 04:20:23 [DEBUG   ] Got command RPT from MCS: ref #533352
2015-11-10 04:20:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:20:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533352  77 57336 15623739 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:20:23 [DEBUG   ] Got command RPT from MCS: ref #533353
2015-11-10 04:20:23 [DEBUG   ] summary = ERROR
2015-11-10 04:20:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533353  13 57336 15623943 A  ERRORERROR'
2015-11-10 04:20:24 [DEBUG   ] Got command RPT from MCS: ref #533354
2015-11-10 04:20:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:20:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533354  77 57336 15624135 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:20:24 [DEBUG   ] Got command RPT from MCS: ref #533355
2015-11-10 04:20:24 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 04:20:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533355  12 57336 15624339 A  ERRORB‘™š'
2015-11-10 04:20:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:20:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:20:51 [DEBUG   ] Finished updating board flags in 1.011 seconds
2015-11-10 04:21:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:21:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:21:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:21:52 [DEBUG   ] Finished updating board flags in 1.593 seconds
2015-11-10 04:21:58 [DEBUG   ] Got command DRX from MCS: ref #533359
2015-11-10 04:21:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:21:58 [DEBUG   ] run_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:21:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533359   8 57336 15718011 A  ERROR'
2015-11-10 04:21:58 [DEBUG   ] Got command DRX from MCS: ref #533360
2015-11-10 04:21:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:21:58 [DEBUG   ] run_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:21:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533360   8 57336 15718015 A  ERROR'
2015-11-10 04:21:58 [INFO    ] Finished starting DRX
2015-11-10 04:21:58 [INFO    ] Finished starting DRX
2015-11-10 04:21:58 [DEBUG   ] Got command BAM from MCS: ref #533361
2015-11-10 04:21:58 [INFO    ] Finished processing BAM request
2015-11-10 04:21:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533361   8 57336 15718211 A  ERROR'
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 04:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 04:21:58 [DEBUG   ] spawn_process: at 2015-11-10 04:21:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 04:22:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:22:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:22:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:22:49 [DEBUG   ] Got command RPT from MCS: ref #533362
2015-11-10 04:22:49 [DEBUG   ] summary = ERROR
2015-11-10 04:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533362  13 57336 15769015 A  ERRORERROR'
2015-11-10 04:22:49 [DEBUG   ] Got command RPT from MCS: ref #533363
2015-11-10 04:22:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533363  77 57336 15769015 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:22:52 [DEBUG   ] Finished updating board flags in 1.349 seconds
2015-11-10 04:23:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:23:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:23:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:23:52 [DEBUG   ] Finished updating board flags in 1.204 seconds
2015-11-10 04:23:58 [DEBUG   ] Got command DRX from MCS: ref #533382
2015-11-10 04:23:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:23:58 [DEBUG   ] run_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:23:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533382   8 57336 15838011 A  ERROR'
2015-11-10 04:23:58 [DEBUG   ] Got command DRX from MCS: ref #533383
2015-11-10 04:23:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:23:58 [DEBUG   ] run_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:23:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533383   8 57336 15838015 A  ERROR'
2015-11-10 04:23:58 [INFO    ] Finished starting DRX
2015-11-10 04:23:58 [INFO    ] Finished starting DRX
2015-11-10 04:23:58 [DEBUG   ] Got command BAM from MCS: ref #533384
2015-11-10 04:23:58 [INFO    ] Finished processing BAM request
2015-11-10 04:23:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533384   8 57336 15838175 A  ERROR'
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 04:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 04:23:58 [DEBUG   ] spawn_process: at 2015-11-10 04:23:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 04:24:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:24:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:24:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:24:52 [DEBUG   ] Finished updating board flags in 1.190 seconds
2015-11-10 04:25:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:25:23 [DEBUG   ] Got command RPT from MCS: ref #533386
2015-11-10 04:25:23 [DEBUG   ] summary = ERROR
2015-11-10 04:25:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533386  13 57336 15923623 A  ERRORERROR'
2015-11-10 04:25:23 [DEBUG   ] Got command RPT from MCS: ref #533387
2015-11-10 04:25:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:25:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533387  77 57336 15923815 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:25:24 [DEBUG   ] Got command RPT from MCS: ref #533388
2015-11-10 04:25:24 [DEBUG   ] summary = ERROR
2015-11-10 04:25:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533388  13 57336 15924023 A  ERRORERROR'
2015-11-10 04:25:24 [DEBUG   ] Got command RPT from MCS: ref #533389
2015-11-10 04:25:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:25:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533389  77 57336 15924223 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:25:24 [DEBUG   ] Got command RPT from MCS: ref #533390
2015-11-10 04:25:24 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 04:25:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533390  12 57336 15924427 A  ERRORB�™š'
2015-11-10 04:25:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:25:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:25:52 [DEBUG   ] Finished updating board flags in 1.037 seconds
2015-11-10 04:25:58 [DEBUG   ] Got command DRX from MCS: ref #533396
2015-11-10 04:25:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:25:58 [DEBUG   ] run_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:25:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533396   8 57336 15958011 A  ERROR'
2015-11-10 04:25:58 [DEBUG   ] Got command DRX from MCS: ref #533397
2015-11-10 04:25:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:25:58 [DEBUG   ] run_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:25:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533397   8 57336 15958015 A  ERROR'
2015-11-10 04:25:58 [INFO    ] Finished starting DRX
2015-11-10 04:25:58 [INFO    ] Finished starting DRX
2015-11-10 04:25:58 [DEBUG   ] Got command BAM from MCS: ref #533398
2015-11-10 04:25:58 [INFO    ] Finished processing BAM request
2015-11-10 04:25:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533398   8 57336 15958199 A  ERROR'
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 04:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 04:25:58 [DEBUG   ] spawn_process: at 2015-11-10 04:25:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 04:26:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:26:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:26:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:26:53 [DEBUG   ] Finished updating board flags in 2.006 seconds
2015-11-10 04:27:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:27:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:27:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:27:49 [DEBUG   ] Got command RPT from MCS: ref #533402
2015-11-10 04:27:49 [DEBUG   ] summary = ERROR
2015-11-10 04:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533402  13 57336 16069011 A  ERRORERROR'
2015-11-10 04:27:49 [DEBUG   ] Got command RPT from MCS: ref #533403
2015-11-10 04:27:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533403  77 57336 16069011 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:27:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:27:52 [DEBUG   ] Finished updating board flags in 1.474 seconds
2015-11-10 04:27:58 [DEBUG   ] Got command DRX from MCS: ref #533419
2015-11-10 04:27:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:27:58 [DEBUG   ] run_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:27:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533419   8 57336 16078015 A  ERROR'
2015-11-10 04:27:58 [DEBUG   ] Got command DRX from MCS: ref #533420
2015-11-10 04:27:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:27:58 [DEBUG   ] run_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:27:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533420   8 57336 16078019 A  ERROR'
2015-11-10 04:27:58 [INFO    ] Finished starting DRX
2015-11-10 04:27:58 [INFO    ] Finished starting DRX
2015-11-10 04:27:58 [DEBUG   ] Got command BAM from MCS: ref #533421
2015-11-10 04:27:58 [INFO    ] Finished processing BAM request
2015-11-10 04:27:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533421   8 57336 16078179 A  ERROR'
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 04:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 04:27:58 [DEBUG   ] spawn_process: at 2015-11-10 04:27:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 04:28:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:28:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:28:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:28:52 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 04:28:52 [DEBUG   ] Finished updating board flags in 1.415 seconds
2015-11-10 04:29:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:29:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:29:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:29:53 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:29:53 [DEBUG   ] Finished updating board flags in 1.952 seconds
2015-11-10 04:29:58 [DEBUG   ] Got command DRX from MCS: ref #533425
2015-11-10 04:29:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:29:58 [DEBUG   ] run_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:29:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533425   8 57336 16198015 A  ERROR'
2015-11-10 04:29:58 [DEBUG   ] Got command DRX from MCS: ref #533426
2015-11-10 04:29:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:29:58 [DEBUG   ] run_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:29:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533426   8 57336 16198019 A  ERROR'
2015-11-10 04:29:58 [INFO    ] Finished starting DRX
2015-11-10 04:29:58 [INFO    ] Finished starting DRX
2015-11-10 04:29:58 [DEBUG   ] Got command BAM from MCS: ref #533427
2015-11-10 04:29:58 [INFO    ] Finished processing BAM request
2015-11-10 04:29:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533427   8 57336 16198143 A  ERROR'
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 04:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 04:29:58 [DEBUG   ] spawn_process: at 2015-11-10 04:29:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 04:30:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:30:23 [DEBUG   ] Got command RPT from MCS: ref #533428
2015-11-10 04:30:23 [DEBUG   ] summary = ERROR
2015-11-10 04:30:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533428  13 57336 16223739 A  ERRORERROR'
2015-11-10 04:30:23 [DEBUG   ] Got command RPT from MCS: ref #533429
2015-11-10 04:30:23 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:30:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533429  77 57336 16223943 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:30:24 [DEBUG   ] Got command RPT from MCS: ref #533430
2015-11-10 04:30:24 [DEBUG   ] summary = ERROR
2015-11-10 04:30:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533430  13 57336 16224147 A  ERRORERROR'
2015-11-10 04:30:24 [DEBUG   ] Got command RPT from MCS: ref #533431
2015-11-10 04:30:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:30:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533431  77 57336 16224339 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:30:24 [DEBUG   ] Got command RPT from MCS: ref #533432
2015-11-10 04:30:24 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 04:30:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533432  12 57336 16224543 A  ERRORB�™š'
2015-11-10 04:30:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:30:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:30:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:30:53 [DEBUG   ] Finished updating board flags in 1.810 seconds
2015-11-10 04:31:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:31:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:31:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:31:52 [DEBUG   ] Finished updating board flags in 1.171 seconds
2015-11-10 04:31:58 [DEBUG   ] Got command DRX from MCS: ref #533433
2015-11-10 04:31:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:31:58 [DEBUG   ] run_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:31:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533433   8 57336 16318007 A  ERROR'
2015-11-10 04:31:58 [INFO    ] Finished starting DRX
2015-11-10 04:31:58 [DEBUG   ] Got command DRX from MCS: ref #533434
2015-11-10 04:31:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:31:58 [DEBUG   ] run_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:31:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533434   8 57336 16318251 A  ERROR'
2015-11-10 04:31:58 [DEBUG   ] Got command BAM from MCS: ref #533435
2015-11-10 04:31:58 [INFO    ] Finished processing BAM request
2015-11-10 04:31:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533435   8 57336 16318255 A  ERROR'
2015-11-10 04:31:58 [INFO    ] Finished starting DRX
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 04:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 04:31:58 [DEBUG   ] spawn_process: at 2015-11-10 04:31:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 04:32:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:32:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:32:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:32:49 [DEBUG   ] Got command RPT from MCS: ref #533439
2015-11-10 04:32:49 [DEBUG   ] summary = ERROR
2015-11-10 04:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533439  13 57336 16369019 A  ERRORERROR'
2015-11-10 04:32:49 [DEBUG   ] Got command RPT from MCS: ref #533440
2015-11-10 04:32:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533440  77 57336 16369019 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:32:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 04:32:53 [DEBUG   ] Finished updating board flags in 1.583 seconds
2015-11-10 04:33:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:33:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:33:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:33:53 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:33:53 [DEBUG   ] Finished updating board flags in 1.387 seconds
2015-11-10 04:33:58 [DEBUG   ] Got command DRX from MCS: ref #533456
2015-11-10 04:33:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:33:58 [DEBUG   ] run_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:33:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533456   8 57336 16438015 A  ERROR'
2015-11-10 04:33:58 [DEBUG   ] Got command DRX from MCS: ref #533457
2015-11-10 04:33:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:33:58 [DEBUG   ] run_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:33:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533457   8 57336 16438019 A  ERROR'
2015-11-10 04:33:58 [INFO    ] Finished starting DRX
2015-11-10 04:33:58 [INFO    ] Finished starting DRX
2015-11-10 04:33:58 [DEBUG   ] Got command BAM from MCS: ref #533458
2015-11-10 04:33:58 [INFO    ] Finished processing BAM request
2015-11-10 04:33:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533458   8 57336 16438179 A  ERROR'
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 04:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 04:33:58 [DEBUG   ] spawn_process: at 2015-11-10 04:33:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 04:34:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:34:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:34:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:34:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 04:34:53 [DEBUG   ] Finished updating board flags in 1.198 seconds
2015-11-10 04:35:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:35:23 [DEBUG   ] Got command RPT from MCS: ref #533460
2015-11-10 04:35:23 [DEBUG   ] summary = ERROR
2015-11-10 04:35:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533460  13 57336 16523825 A  ERRORERROR'
2015-11-10 04:35:24 [DEBUG   ] Got command RPT from MCS: ref #533461
2015-11-10 04:35:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:35:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533461  77 57336 16524027 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:35:24 [DEBUG   ] Got command RPT from MCS: ref #533462
2015-11-10 04:35:24 [DEBUG   ] summary = ERROR
2015-11-10 04:35:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533462  13 57336 16524423 A  ERRORERROR'
2015-11-10 04:35:24 [DEBUG   ] Got command RPT from MCS: ref #533463
2015-11-10 04:35:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:35:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533463  77 57336 16524627 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:35:24 [DEBUG   ] Got command RPT from MCS: ref #533464
2015-11-10 04:35:24 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 04:35:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533464  12 57336 16524831 A  ERRORB�™š'
2015-11-10 04:35:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:35:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:35:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:35:53 [DEBUG   ] Finished updating board flags in 1.012 seconds
2015-11-10 04:35:58 [DEBUG   ] Got command DRX from MCS: ref #533473
2015-11-10 04:35:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:35:58 [DEBUG   ] run_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:35:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533473   8 57336 16558015 A  ERROR'
2015-11-10 04:35:58 [DEBUG   ] Got command DRX from MCS: ref #533474
2015-11-10 04:35:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:35:58 [DEBUG   ] run_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:35:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533474   8 57336 16558019 A  ERROR'
2015-11-10 04:35:58 [INFO    ] Finished starting DRX
2015-11-10 04:35:58 [INFO    ] Finished starting DRX
2015-11-10 04:35:58 [DEBUG   ] Got command BAM from MCS: ref #533475
2015-11-10 04:35:58 [INFO    ] Finished processing BAM request
2015-11-10 04:35:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533475   8 57336 16558227 A  ERROR'
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 04:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 04:35:58 [DEBUG   ] spawn_process: at 2015-11-10 04:35:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 04:36:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:36:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:36:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:36:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 04:36:53 [DEBUG   ] Finished updating board flags in 1.013 seconds
2015-11-10 04:37:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:37:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:37:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:37:49 [DEBUG   ] Got command RPT from MCS: ref #533476
2015-11-10 04:37:49 [DEBUG   ] summary = ERROR
2015-11-10 04:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533476  13 57336 16669011 A  ERRORERROR'
2015-11-10 04:37:49 [DEBUG   ] Got command RPT from MCS: ref #533477
2015-11-10 04:37:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533477  77 57336 16669011 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:37:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 04:37:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:37:54 [DEBUG   ] Finished updating board flags in 2.038 seconds
2015-11-10 04:37:58 [DEBUG   ] Got command DRX from MCS: ref #533493
2015-11-10 04:37:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:37:58 [DEBUG   ] run_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:37:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533493   8 57336 16678015 A  ERROR'
2015-11-10 04:37:58 [DEBUG   ] Got command DRX from MCS: ref #533494
2015-11-10 04:37:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:37:58 [DEBUG   ] run_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:37:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533494   8 57336 16678019 A  ERROR'
2015-11-10 04:37:58 [INFO    ] Finished starting DRX
2015-11-10 04:37:58 [INFO    ] Finished starting DRX
2015-11-10 04:37:58 [DEBUG   ] Got command BAM from MCS: ref #533495
2015-11-10 04:37:58 [INFO    ] Finished processing BAM request
2015-11-10 04:37:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533495   8 57336 16678191 A  ERROR'
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 04:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 04:37:58 [DEBUG   ] spawn_process: at 2015-11-10 04:37:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 04:38:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:38:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:38:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:38:53 [DEBUG   ] Finished updating board flags in 1.385 seconds
2015-11-10 04:39:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:39:28 [DEBUG   ] Finished updating ant stats in 1.600 seconds
2015-11-10 04:39:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:39:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 04:39:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:39:54 [DEBUG   ] Finished updating board flags in 1.797 seconds
2015-11-10 04:39:58 [DEBUG   ] Got command DRX from MCS: ref #533499
2015-11-10 04:39:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:39:58 [DEBUG   ] run_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533499   8 57336 16798015 A  ERROR'
2015-11-10 04:39:58 [DEBUG   ] Got command DRX from MCS: ref #533500
2015-11-10 04:39:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:39:58 [DEBUG   ] run_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533500   8 57336 16798019 A  ERROR'
2015-11-10 04:39:58 [INFO    ] Finished starting DRX
2015-11-10 04:39:58 [INFO    ] Finished starting DRX
2015-11-10 04:39:58 [DEBUG   ] Got command BAM from MCS: ref #533501
2015-11-10 04:39:58 [INFO    ] Finished processing BAM request
2015-11-10 04:39:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533501   8 57336 16798179 A  ERROR'
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 04:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 04:39:58 [DEBUG   ] spawn_process: at 2015-11-10 04:39:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 04:40:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:40:23 [DEBUG   ] Got command RPT from MCS: ref #533502
2015-11-10 04:40:23 [DEBUG   ] summary = ERROR
2015-11-10 04:40:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533502  13 57336 16823923 A  ERRORERROR'
2015-11-10 04:40:24 [DEBUG   ] Got command RPT from MCS: ref #533503
2015-11-10 04:40:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:40:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533503  77 57336 16824127 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:40:24 [DEBUG   ] Got command RPT from MCS: ref #533504
2015-11-10 04:40:24 [DEBUG   ] summary = ERROR
2015-11-10 04:40:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533504  13 57336 16824331 A  ERRORERROR'
2015-11-10 04:40:24 [DEBUG   ] Got command RPT from MCS: ref #533505
2015-11-10 04:40:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:40:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533505  77 57336 16824535 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:40:24 [DEBUG   ] Got command RPT from MCS: ref #533506
2015-11-10 04:40:24 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 04:40:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533506  12 57336 16824727 A  ERRORB�™š'
2015-11-10 04:40:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:40:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:40:53 [DEBUG   ] Finished updating board flags in 0.999 seconds
2015-11-10 04:41:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:41:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:41:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:41:54 [DEBUG   ] Finished updating board flags in 1.374 seconds
2015-11-10 04:41:58 [DEBUG   ] Got command DRX from MCS: ref #533510
2015-11-10 04:41:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:41:58 [DEBUG   ] run_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:41:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533510   8 57336 16918011 A  ERROR'
2015-11-10 04:41:58 [DEBUG   ] Got command DRX from MCS: ref #533511
2015-11-10 04:41:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:41:58 [DEBUG   ] run_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:41:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533511   8 57336 16918015 A  ERROR'
2015-11-10 04:41:58 [INFO    ] Finished starting DRX
2015-11-10 04:41:58 [INFO    ] Finished starting DRX
2015-11-10 04:41:58 [DEBUG   ] Got command BAM from MCS: ref #533512
2015-11-10 04:41:58 [INFO    ] Finished processing BAM request
2015-11-10 04:41:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533512   8 57336 16918239 A  ERROR'
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 04:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 04:41:58 [DEBUG   ] spawn_process: at 2015-11-10 04:41:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 04:42:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:42:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:42:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:42:49 [DEBUG   ] Got command RPT from MCS: ref #533513
2015-11-10 04:42:49 [DEBUG   ] summary = ERROR
2015-11-10 04:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533513  13 57336 16969015 A  ERRORERROR'
2015-11-10 04:42:49 [DEBUG   ] Got command RPT from MCS: ref #533514
2015-11-10 04:42:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533514  77 57336 16969015 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:42:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 04:42:54 [DEBUG   ] Finished updating board flags in 1.188 seconds
2015-11-10 04:43:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:43:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:43:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:43:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 04:43:54 [DEBUG   ] Finished updating board flags in 1.077 seconds
2015-11-10 04:43:58 [DEBUG   ] Got command DRX from MCS: ref #533530
2015-11-10 04:43:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:43:58 [DEBUG   ] run_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:43:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533530   8 57336 17038007 A  ERROR'
2015-11-10 04:43:58 [INFO    ] Finished starting DRX
2015-11-10 04:43:58 [DEBUG   ] Got command DRX from MCS: ref #533531
2015-11-10 04:43:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:43:58 [DEBUG   ] run_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:43:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533531   8 57336 17038167 A  ERROR'
2015-11-10 04:43:58 [DEBUG   ] Got command BAM from MCS: ref #533532
2015-11-10 04:43:58 [INFO    ] Finished processing BAM request
2015-11-10 04:43:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533532   8 57336 17038171 A  ERROR'
2015-11-10 04:43:58 [INFO    ] Finished starting DRX
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 04:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 04:43:58 [DEBUG   ] spawn_process: at 2015-11-10 04:43:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 04:44:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:44:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:44:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:44:55 [DEBUG   ] Finished updating board flags in 2.001 seconds
2015-11-10 04:45:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:45:24 [DEBUG   ] Got command RPT from MCS: ref #533537
2015-11-10 04:45:24 [DEBUG   ] summary = ERROR
2015-11-10 04:45:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533537  13 57336 17124051 A  ERRORERROR'
2015-11-10 04:45:24 [DEBUG   ] Got command RPT from MCS: ref #533538
2015-11-10 04:45:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:45:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533538  77 57336 17124243 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:45:24 [DEBUG   ] Got command RPT from MCS: ref #533539
2015-11-10 04:45:24 [DEBUG   ] summary = ERROR
2015-11-10 04:45:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533539  13 57336 17124447 A  ERRORERROR'
2015-11-10 04:45:24 [DEBUG   ] Got command RPT from MCS: ref #533540
2015-11-10 04:45:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:45:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533540  77 57336 17124651 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:45:24 [DEBUG   ] Got command RPT from MCS: ref #533541
2015-11-10 04:45:24 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 04:45:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533541  12 57336 17124855 A  ERRORB�™š'
2015-11-10 04:45:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:45:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:45:55 [DEBUG   ] Finished updating board flags in 1.946 seconds
2015-11-10 04:45:58 [DEBUG   ] Got command DRX from MCS: ref #533547
2015-11-10 04:45:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:45:58 [DEBUG   ] run_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:45:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533547   8 57336 17158007 A  ERROR'
2015-11-10 04:45:58 [INFO    ] Finished starting DRX
2015-11-10 04:45:58 [DEBUG   ] Got command DRX from MCS: ref #533548
2015-11-10 04:45:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:45:58 [DEBUG   ] run_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:45:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533548   8 57336 17158203 A  ERROR'
2015-11-10 04:45:58 [DEBUG   ] Got command BAM from MCS: ref #533549
2015-11-10 04:45:58 [INFO    ] Finished processing BAM request
2015-11-10 04:45:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533549   8 57336 17158207 A  ERROR'
2015-11-10 04:45:58 [INFO    ] Finished starting DRX
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 04:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 04:45:58 [DEBUG   ] spawn_process: at 2015-11-10 04:45:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 04:46:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:46:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:46:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:46:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 04:46:54 [DEBUG   ] Finished updating board flags in 1.249 seconds
2015-11-10 04:47:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:47:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:47:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:47:49 [DEBUG   ] Got command RPT from MCS: ref #533553
2015-11-10 04:47:49 [DEBUG   ] summary = ERROR
2015-11-10 04:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533553  13 57336 17269019 A  ERRORERROR'
2015-11-10 04:47:49 [DEBUG   ] Got command RPT from MCS: ref #533554
2015-11-10 04:47:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533554  77 57336 17269019 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:47:54 [DEBUG   ] Finished updating board flags in 1.226 seconds
2015-11-10 04:47:58 [DEBUG   ] Got command DRX from MCS: ref #533570
2015-11-10 04:47:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:47:58 [DEBUG   ] run_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:47:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533570   8 57336 17278011 A  ERROR'
2015-11-10 04:47:58 [DEBUG   ] Got command DRX from MCS: ref #533571
2015-11-10 04:47:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:47:58 [DEBUG   ] run_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:47:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533571   8 57336 17278015 A  ERROR'
2015-11-10 04:47:58 [INFO    ] Finished starting DRX
2015-11-10 04:47:58 [INFO    ] Finished starting DRX
2015-11-10 04:47:58 [DEBUG   ] Got command BAM from MCS: ref #533572
2015-11-10 04:47:58 [INFO    ] Finished processing BAM request
2015-11-10 04:47:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533572   8 57336 17278175 A  ERROR'
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 04:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 04:47:58 [DEBUG   ] spawn_process: at 2015-11-10 04:47:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 04:48:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:48:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:48:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:48:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 04:48:55 [DEBUG   ] Finished updating board flags in 1.736 seconds
2015-11-10 04:49:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:49:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:49:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:49:54 [DEBUG   ] Finished updating board flags in 1.025 seconds
2015-11-10 04:49:58 [DEBUG   ] Got command DRX from MCS: ref #533573
2015-11-10 04:49:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:49:58 [DEBUG   ] run_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:49:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533573   8 57336 17398011 A  ERROR'
2015-11-10 04:49:58 [DEBUG   ] Got command DRX from MCS: ref #533574
2015-11-10 04:49:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:49:58 [DEBUG   ] run_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:49:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533574   8 57336 17398015 A  ERROR'
2015-11-10 04:49:58 [INFO    ] Finished starting DRX
2015-11-10 04:49:58 [INFO    ] Finished starting DRX
2015-11-10 04:49:58 [DEBUG   ] Got command BAM from MCS: ref #533575
2015-11-10 04:49:58 [INFO    ] Finished processing BAM request
2015-11-10 04:49:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533575   8 57336 17398247 A  ERROR'
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 04:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 04:49:58 [DEBUG   ] spawn_process: at 2015-11-10 04:49:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 04:50:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:50:24 [DEBUG   ] Got command RPT from MCS: ref #533576
2015-11-10 04:50:24 [DEBUG   ] summary = ERROR
2015-11-10 04:50:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533576  13 57336 17424163 A  ERRORERROR'
2015-11-10 04:50:24 [DEBUG   ] Got command RPT from MCS: ref #533577
2015-11-10 04:50:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:50:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533577  77 57336 17424355 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:50:24 [DEBUG   ] Got command RPT from MCS: ref #533578
2015-11-10 04:50:24 [DEBUG   ] summary = ERROR
2015-11-10 04:50:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533578  13 57336 17424559 A  ERRORERROR'
2015-11-10 04:50:24 [DEBUG   ] Got command RPT from MCS: ref #533579
2015-11-10 04:50:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:50:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533579  77 57336 17424763 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:50:24 [DEBUG   ] Got command RPT from MCS: ref #533580
2015-11-10 04:50:24 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 04:50:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533580  12 57336 17424955 A  ERRORB�ÌÍ'
2015-11-10 04:50:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:50:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:50:54 [DEBUG   ] Finished updating board flags in 1.016 seconds
2015-11-10 04:51:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:51:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:51:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:51:55 [DEBUG   ] Finished updating board flags in 1.607 seconds
2015-11-10 04:51:58 [DEBUG   ] Got command DRX from MCS: ref #533584
2015-11-10 04:51:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:51:58 [DEBUG   ] run_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:51:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533584   8 57336 17518015 A  ERROR'
2015-11-10 04:51:58 [DEBUG   ] Got command DRX from MCS: ref #533585
2015-11-10 04:51:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:51:58 [DEBUG   ] run_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:51:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533585   8 57336 17518019 A  ERROR'
2015-11-10 04:51:58 [INFO    ] Finished starting DRX
2015-11-10 04:51:58 [INFO    ] Finished starting DRX
2015-11-10 04:51:58 [DEBUG   ] Got command BAM from MCS: ref #533586
2015-11-10 04:51:58 [INFO    ] Finished processing BAM request
2015-11-10 04:51:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533586   8 57336 17518203 A  ERROR'
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 04:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 04:51:58 [DEBUG   ] spawn_process: at 2015-11-10 04:51:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 04:52:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:52:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:52:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:52:49 [DEBUG   ] Got command RPT from MCS: ref #533587
2015-11-10 04:52:49 [DEBUG   ] summary = ERROR
2015-11-10 04:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533587  13 57336 17569015 A  ERRORERROR'
2015-11-10 04:52:49 [DEBUG   ] Got command RPT from MCS: ref #533588
2015-11-10 04:52:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533588  77 57336 17569016 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:52:55 [DEBUG   ] Finished updating board flags in 1.547 seconds
2015-11-10 04:53:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:53:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 04:53:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 04:53:55 [DEBUG   ] Finished updating board flags in 1.415 seconds
2015-11-10 04:53:58 [DEBUG   ] Got command DRX from MCS: ref #533607
2015-11-10 04:53:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:53:58 [DEBUG   ] run_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:53:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533607   8 57336 17638007 A  ERROR'
2015-11-10 04:53:58 [INFO    ] Finished starting DRX
2015-11-10 04:53:58 [DEBUG   ] Got command DRX from MCS: ref #533608
2015-11-10 04:53:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:53:58 [DEBUG   ] run_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:53:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533608   8 57336 17638179 A  ERROR'
2015-11-10 04:53:58 [DEBUG   ] Got command BAM from MCS: ref #533609
2015-11-10 04:53:58 [INFO    ] Finished processing BAM request
2015-11-10 04:53:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533609   8 57336 17638183 A  ERROR'
2015-11-10 04:53:58 [INFO    ] Finished starting DRX
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 04:53:58 [DEBUG   ] spawn_process: at 2015-11-10 04:53:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 04:54:07 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2015-11-10 04:54:27 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 04:54:47 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 04:54:55 [DEBUG   ] Finished updating board flags in 1.379 seconds
2015-11-10 04:55:07 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 04:55:24 [DEBUG   ] Got command RPT from MCS: ref #533611
2015-11-10 04:55:24 [DEBUG   ] summary = ERROR
2015-11-10 04:55:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533611  13 57336 17724247 A  ERRORERROR'
2015-11-10 04:55:24 [DEBUG   ] Got command RPT from MCS: ref #533612
2015-11-10 04:55:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:55:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533612  77 57336 17724451 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:55:24 [DEBUG   ] Got command RPT from MCS: ref #533613
2015-11-10 04:55:24 [DEBUG   ] summary = ERROR
2015-11-10 04:55:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533613  13 57336 17724655 A  ERRORERROR'
2015-11-10 04:55:24 [DEBUG   ] Got command RPT from MCS: ref #533614
2015-11-10 04:55:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:55:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533614  77 57336 17724859 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:55:25 [DEBUG   ] Got command RPT from MCS: ref #533615
2015-11-10 04:55:25 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 04:55:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533615  12 57336 17725051 A  ERRORB‘™š'
2015-11-10 04:55:27 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 04:55:47 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 04:55:55 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:55:55 [DEBUG   ] Finished updating board flags in 1.211 seconds
2015-11-10 04:55:58 [DEBUG   ] Got command DRX from MCS: ref #533621
2015-11-10 04:55:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:55:58 [DEBUG   ] run_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:55:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533621   8 57336 17758007 A  ERROR'
2015-11-10 04:55:58 [INFO    ] Finished starting DRX
2015-11-10 04:55:58 [DEBUG   ] Got command DRX from MCS: ref #533622
2015-11-10 04:55:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:55:58 [DEBUG   ] run_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:55:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533622   8 57336 17758143 A  ERROR'
2015-11-10 04:55:58 [DEBUG   ] Got command BAM from MCS: ref #533623
2015-11-10 04:55:58 [INFO    ] Finished processing BAM request
2015-11-10 04:55:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533623   8 57336 17758147 A  ERROR'
2015-11-10 04:55:58 [INFO    ] Finished starting DRX
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 04:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 04:55:58 [DEBUG   ] spawn_process: at 2015-11-10 04:55:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 04:56:07 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 04:56:27 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 04:56:47 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 04:56:55 [DEBUG   ] Finished updating board flags in 1.212 seconds
2015-11-10 04:57:07 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 04:57:27 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 04:57:47 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 04:57:49 [DEBUG   ] Got command RPT from MCS: ref #533627
2015-11-10 04:57:49 [DEBUG   ] summary = ERROR
2015-11-10 04:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533627  13 57336 17869019 A  ERRORERROR'
2015-11-10 04:57:49 [DEBUG   ] Got command RPT from MCS: ref #533628
2015-11-10 04:57:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 04:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533628  77 57336 17869019 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 04:57:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 04:57:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 04:57:55 [DEBUG   ] Finished updating board flags in 1.185 seconds
2015-11-10 04:57:58 [DEBUG   ] Got command DRX from MCS: ref #533644
2015-11-10 04:57:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:57:58 [DEBUG   ] run_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:57:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533644   8 57336 17878015 A  ERROR'
2015-11-10 04:57:58 [DEBUG   ] Got command DRX from MCS: ref #533645
2015-11-10 04:57:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:57:58 [DEBUG   ] run_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:57:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533645   8 57336 17878019 A  ERROR'
2015-11-10 04:57:58 [INFO    ] Finished starting DRX
2015-11-10 04:57:58 [INFO    ] Finished starting DRX
2015-11-10 04:57:58 [DEBUG   ] Got command BAM from MCS: ref #533646
2015-11-10 04:57:58 [INFO    ] Finished processing BAM request
2015-11-10 04:57:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533646   8 57336 17878203 A  ERROR'
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 04:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 04:57:58 [DEBUG   ] spawn_process: at 2015-11-10 04:57:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 04:58:07 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 04:58:28 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 04:58:47 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 04:58:55 [DEBUG   ] Finished updating board flags in 1.077 seconds
2015-11-10 04:59:07 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 04:59:27 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 04:59:47 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 04:59:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 04:59:56 [DEBUG   ] Finished updating board flags in 2.007 seconds
2015-11-10 04:59:58 [DEBUG   ] Got command DRX from MCS: ref #533650
2015-11-10 04:59:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:59:58 [DEBUG   ] run_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 04:59:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533650   8 57336 17998007 A  ERROR'
2015-11-10 04:59:58 [INFO    ] Finished starting DRX
2015-11-10 04:59:58 [DEBUG   ] Got command DRX from MCS: ref #533651
2015-11-10 04:59:58 [INFO    ] sending drx command to lwa16
2015-11-10 04:59:58 [DEBUG   ] run_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 04:59:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 04:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533651   8 57336 17998167 A  ERROR'
2015-11-10 04:59:58 [DEBUG   ] Got command BAM from MCS: ref #533652
2015-11-10 04:59:58 [INFO    ] Finished processing BAM request
2015-11-10 04:59:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 04:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533652   8 57336 17998171 A  ERROR'
2015-11-10 04:59:58 [INFO    ] Finished starting DRX
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 04:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 04:59:58 [DEBUG   ] spawn_process: at 2015-11-10 04:59:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 05:00:07 [DEBUG   ] Finished updating ant stats in 0.397 seconds
2015-11-10 05:00:24 [DEBUG   ] Got command RPT from MCS: ref #533653
2015-11-10 05:00:24 [DEBUG   ] summary = ERROR
2015-11-10 05:00:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533653  13 57336 18024359 A  ERRORERROR'
2015-11-10 05:00:24 [DEBUG   ] Got command RPT from MCS: ref #533654
2015-11-10 05:00:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:00:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533654  77 57336 18024551 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:00:24 [DEBUG   ] Got command RPT from MCS: ref #533655
2015-11-10 05:00:24 [DEBUG   ] summary = ERROR
2015-11-10 05:00:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533655  13 57336 18024755 A  ERRORERROR'
2015-11-10 05:00:24 [DEBUG   ] Got command RPT from MCS: ref #533656
2015-11-10 05:00:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:00:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533656  77 57336 18024959 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:00:25 [DEBUG   ] Got command RPT from MCS: ref #533657
2015-11-10 05:00:25 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 05:00:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533657  12 57336 18025163 A  ERRORB�™š'
2015-11-10 05:00:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 05:00:48 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:00:55 [DEBUG   ] Finished updating board flags in 1.387 seconds
2015-11-10 05:01:08 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:01:28 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:01:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 05:01:55 [DEBUG   ] Finished updating board flags in 1.202 seconds
2015-11-10 05:01:58 [DEBUG   ] Got command DRX from MCS: ref #533658
2015-11-10 05:01:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:01:58 [DEBUG   ] run_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:01:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533658   8 57336 18118007 A  ERROR'
2015-11-10 05:01:58 [INFO    ] Finished starting DRX
2015-11-10 05:01:58 [DEBUG   ] Got command DRX from MCS: ref #533659
2015-11-10 05:01:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:01:58 [DEBUG   ] run_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:01:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533659   8 57336 18118131 A  ERROR'
2015-11-10 05:01:58 [DEBUG   ] Got command BAM from MCS: ref #533660
2015-11-10 05:01:58 [INFO    ] Finished processing BAM request
2015-11-10 05:01:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533660   8 57336 18118135 A  ERROR'
2015-11-10 05:01:58 [INFO    ] Finished starting DRX
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 05:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 05:01:58 [DEBUG   ] spawn_process: at 2015-11-10 05:01:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 05:02:08 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:02:28 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:02:48 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:02:49 [DEBUG   ] Got command RPT from MCS: ref #533664
2015-11-10 05:02:49 [DEBUG   ] summary = ERROR
2015-11-10 05:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533664  13 57336 18169019 A  ERRORERROR'
2015-11-10 05:02:49 [DEBUG   ] Got command RPT from MCS: ref #533665
2015-11-10 05:02:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533665  77 57336 18169019 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:02:55 [DEBUG   ] Finished updating board flags in 1.212 seconds
2015-11-10 05:03:08 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:03:28 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:03:48 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:03:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 05:03:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:03:55 [DEBUG   ] Finished updating board flags in 1.165 seconds
2015-11-10 05:03:58 [DEBUG   ] Got command DRX from MCS: ref #533681
2015-11-10 05:03:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:03:58 [DEBUG   ] run_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:03:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533681   8 57336 18238015 A  ERROR'
2015-11-10 05:03:58 [DEBUG   ] Got command DRX from MCS: ref #533682
2015-11-10 05:03:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:03:58 [DEBUG   ] run_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:03:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533682   8 57336 18238019 A  ERROR'
2015-11-10 05:03:58 [INFO    ] Finished starting DRX
2015-11-10 05:03:58 [INFO    ] Finished starting DRX
2015-11-10 05:03:58 [DEBUG   ] Got command BAM from MCS: ref #533683
2015-11-10 05:03:58 [INFO    ] Finished processing BAM request
2015-11-10 05:03:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533683   8 57336 18238179 A  ERROR'
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 05:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 05:03:58 [DEBUG   ] spawn_process: at 2015-11-10 05:03:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 05:04:08 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:04:28 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:04:48 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 05:04:55 [DEBUG   ] Finished updating board flags in 1.030 seconds
2015-11-10 05:05:08 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:05:24 [DEBUG   ] Got command RPT from MCS: ref #533685
2015-11-10 05:05:24 [DEBUG   ] summary = ERROR
2015-11-10 05:05:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533685  13 57336 18324439 A  ERRORERROR'
2015-11-10 05:05:24 [DEBUG   ] Got command RPT from MCS: ref #533686
2015-11-10 05:05:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:05:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533686  77 57336 18324643 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:05:24 [DEBUG   ] Got command RPT from MCS: ref #533687
2015-11-10 05:05:24 [DEBUG   ] summary = ERROR
2015-11-10 05:05:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533687  13 57336 18324847 A  ERRORERROR'
2015-11-10 05:05:25 [DEBUG   ] Got command RPT from MCS: ref #533688
2015-11-10 05:05:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:05:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533688  77 57336 18325039 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:05:25 [DEBUG   ] Got command RPT from MCS: ref #533689
2015-11-10 05:05:25 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 05:05:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533689  12 57336 18325243 A  ERRORB�™š'
2015-11-10 05:05:28 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:05:48 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:05:55 [DEBUG   ] Finished updating board flags in 0.983 seconds
2015-11-10 05:05:58 [DEBUG   ] Got command DRX from MCS: ref #533698
2015-11-10 05:05:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:05:58 [DEBUG   ] run_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:05:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533698   8 57336 18358011 A  ERROR'
2015-11-10 05:05:58 [DEBUG   ] Got command DRX from MCS: ref #533699
2015-11-10 05:05:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:05:58 [DEBUG   ] run_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:05:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533699   8 57336 18358015 A  ERROR'
2015-11-10 05:05:58 [INFO    ] Finished starting DRX
2015-11-10 05:05:58 [INFO    ] Finished starting DRX
2015-11-10 05:05:58 [DEBUG   ] Got command BAM from MCS: ref #533700
2015-11-10 05:05:58 [INFO    ] Finished processing BAM request
2015-11-10 05:05:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533700   8 57336 18358187 A  ERROR'
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 05:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 05:05:58 [DEBUG   ] spawn_process: at 2015-11-10 05:05:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 05:06:08 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:06:28 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:06:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:06:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 05:06:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 05:06:56 [DEBUG   ] Finished updating board flags in 1.362 seconds
2015-11-10 05:07:08 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:07:28 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:07:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:07:49 [DEBUG   ] Got command RPT from MCS: ref #533701
2015-11-10 05:07:49 [DEBUG   ] summary = ERROR
2015-11-10 05:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533701  13 57336 18469027 A  ERRORERROR'
2015-11-10 05:07:49 [DEBUG   ] Got command RPT from MCS: ref #533702
2015-11-10 05:07:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533702  77 57336 18469027 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:07:56 [DEBUG   ] Finished updating board flags in 1.186 seconds
2015-11-10 05:07:58 [DEBUG   ] Got command DRX from MCS: ref #533718
2015-11-10 05:07:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:07:58 [DEBUG   ] run_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:07:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533718   8 57336 18478011 A  ERROR'
2015-11-10 05:07:58 [DEBUG   ] Got command DRX from MCS: ref #533719
2015-11-10 05:07:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:07:58 [DEBUG   ] run_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:07:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533719   8 57336 18478015 A  ERROR'
2015-11-10 05:07:58 [INFO    ] Finished starting DRX
2015-11-10 05:07:58 [INFO    ] Finished starting DRX
2015-11-10 05:07:58 [DEBUG   ] Got command BAM from MCS: ref #533720
2015-11-10 05:07:58 [INFO    ] Finished processing BAM request
2015-11-10 05:07:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533720   8 57336 18478163 A  ERROR'
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 05:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 05:07:58 [DEBUG   ] spawn_process: at 2015-11-10 05:07:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 05:08:08 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:08:28 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:08:48 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:08:56 [DEBUG   ] Finished updating board flags in 1.388 seconds
2015-11-10 05:09:08 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:09:28 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:09:49 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:09:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 05:09:57 [DEBUG   ] Finished updating board flags in 1.830 seconds
2015-11-10 05:09:58 [DEBUG   ] Got command DRX from MCS: ref #533724
2015-11-10 05:09:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:09:58 [DEBUG   ] run_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:09:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533724   8 57336 18598015 A  ERROR'
2015-11-10 05:09:58 [DEBUG   ] Got command DRX from MCS: ref #533725
2015-11-10 05:09:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:09:58 [DEBUG   ] run_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:09:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533725   8 57336 18598019 A  ERROR'
2015-11-10 05:09:58 [INFO    ] Finished starting DRX
2015-11-10 05:09:58 [INFO    ] Finished starting DRX
2015-11-10 05:09:58 [DEBUG   ] Got command BAM from MCS: ref #533726
2015-11-10 05:09:58 [INFO    ] Finished processing BAM request
2015-11-10 05:09:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533726   8 57336 18598179 A  ERROR'
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 05:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 05:09:58 [DEBUG   ] spawn_process: at 2015-11-10 05:09:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 05:10:09 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:10:24 [DEBUG   ] Got command RPT from MCS: ref #533727
2015-11-10 05:10:24 [DEBUG   ] summary = ERROR
2015-11-10 05:10:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533727  13 57336 18624543 A  ERRORERROR'
2015-11-10 05:10:24 [DEBUG   ] Got command RPT from MCS: ref #533728
2015-11-10 05:10:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:10:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533728  77 57336 18624939 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:10:25 [DEBUG   ] Got command RPT from MCS: ref #533729
2015-11-10 05:10:25 [DEBUG   ] summary = ERROR
2015-11-10 05:10:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533729  13 57336 18625143 A  ERRORERROR'
2015-11-10 05:10:25 [DEBUG   ] Got command RPT from MCS: ref #533730
2015-11-10 05:10:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:10:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533730  77 57336 18625347 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:10:25 [DEBUG   ] Got command RPT from MCS: ref #533731
2015-11-10 05:10:25 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 05:10:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533731  12 57336 18625539 A  ERRORB�™š'
2015-11-10 05:10:29 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:10:49 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:10:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:10:57 [DEBUG   ] Finished updating board flags in 1.865 seconds
2015-11-10 05:11:09 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:11:29 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:11:49 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:11:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:11:57 [DEBUG   ] Finished updating board flags in 1.734 seconds
2015-11-10 05:11:58 [DEBUG   ] Got command DRX from MCS: ref #533735
2015-11-10 05:11:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:11:58 [DEBUG   ] run_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:11:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533735   8 57336 18718015 A  ERROR'
2015-11-10 05:11:58 [DEBUG   ] Got command DRX from MCS: ref #533736
2015-11-10 05:11:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:11:58 [DEBUG   ] run_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:11:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533736   8 57336 18718019 A  ERROR'
2015-11-10 05:11:58 [INFO    ] Finished starting DRX
2015-11-10 05:11:58 [INFO    ] Finished starting DRX
2015-11-10 05:11:58 [DEBUG   ] Got command BAM from MCS: ref #533737
2015-11-10 05:11:58 [INFO    ] Finished processing BAM request
2015-11-10 05:11:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533737   8 57336 18718191 A  ERROR'
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 05:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 05:11:58 [DEBUG   ] spawn_process: at 2015-11-10 05:11:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 05:12:09 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:12:29 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:12:49 [DEBUG   ] Got command RPT from MCS: ref #533738
2015-11-10 05:12:49 [DEBUG   ] summary = ERROR
2015-11-10 05:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533738  13 57336 18769019 A  ERRORERROR'
2015-11-10 05:12:49 [DEBUG   ] Got command RPT from MCS: ref #533739
2015-11-10 05:12:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533739  77 57336 18769019 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:12:49 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:12:56 [DEBUG   ] Finished updating board flags in 1.048 seconds
2015-11-10 05:13:09 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:13:29 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:13:49 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:13:56 [DEBUG   ] Finished updating board flags in 1.036 seconds
2015-11-10 05:13:58 [DEBUG   ] Got command DRX from MCS: ref #533755
2015-11-10 05:13:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:13:58 [DEBUG   ] run_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:13:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533755   8 57336 18838007 A  ERROR'
2015-11-10 05:13:58 [INFO    ] Finished starting DRX
2015-11-10 05:13:58 [DEBUG   ] Got command DRX from MCS: ref #533756
2015-11-10 05:13:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:13:58 [DEBUG   ] run_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:13:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533756   8 57336 18838179 A  ERROR'
2015-11-10 05:13:58 [DEBUG   ] Got command BAM from MCS: ref #533757
2015-11-10 05:13:58 [INFO    ] Finished processing BAM request
2015-11-10 05:13:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533757   8 57336 18838183 A  ERROR'
2015-11-10 05:13:58 [INFO    ] Finished starting DRX
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 05:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 05:13:58 [DEBUG   ] spawn_process: at 2015-11-10 05:13:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 05:14:09 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:14:29 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:14:49 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:14:56 [DEBUG   ] Finished updating board flags in 0.961 seconds
2015-11-10 05:15:09 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:15:24 [DEBUG   ] Got command RPT from MCS: ref #533762
2015-11-10 05:15:24 [DEBUG   ] summary = ERROR
2015-11-10 05:15:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533762  13 57336 18924623 A  ERRORERROR'
2015-11-10 05:15:24 [DEBUG   ] Got command RPT from MCS: ref #533763
2015-11-10 05:15:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:15:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533763  77 57336 18924827 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:15:25 [DEBUG   ] Got command RPT from MCS: ref #533764
2015-11-10 05:15:25 [DEBUG   ] summary = ERROR
2015-11-10 05:15:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533764  13 57336 18925019 A  ERRORERROR'
2015-11-10 05:15:25 [DEBUG   ] Got command RPT from MCS: ref #533765
2015-11-10 05:15:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:15:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533765  77 57336 18925223 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:15:25 [DEBUG   ] Got command RPT from MCS: ref #533766
2015-11-10 05:15:25 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 05:15:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533766  12 57336 18925427 A  ERRORB�™š'
2015-11-10 05:15:29 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:15:49 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:15:57 [DEBUG   ] Finished updating board flags in 1.361 seconds
2015-11-10 05:15:58 [DEBUG   ] Got command DRX from MCS: ref #533772
2015-11-10 05:15:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:15:58 [DEBUG   ] run_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:15:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533772   8 57336 18958007 A  ERROR'
2015-11-10 05:15:58 [INFO    ] Finished starting DRX
2015-11-10 05:15:58 [DEBUG   ] Got command DRX from MCS: ref #533773
2015-11-10 05:15:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:15:58 [DEBUG   ] run_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:15:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533773   8 57336 18958191 A  ERROR'
2015-11-10 05:15:58 [DEBUG   ] Got command BAM from MCS: ref #533774
2015-11-10 05:15:58 [INFO    ] Finished processing BAM request
2015-11-10 05:15:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533774   8 57336 18958195 A  ERROR'
2015-11-10 05:15:58 [INFO    ] Finished starting DRX
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 05:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 05:15:58 [DEBUG   ] spawn_process: at 2015-11-10 05:15:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 05:16:09 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:16:29 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:16:49 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:16:57 [DEBUG   ] Finished updating board flags in 1.206 seconds
2015-11-10 05:17:09 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:17:29 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:17:49 [DEBUG   ] Got command RPT from MCS: ref #533778
2015-11-10 05:17:49 [DEBUG   ] summary = ERROR
2015-11-10 05:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533778  13 57336 19069019 A  ERRORERROR'
2015-11-10 05:17:49 [DEBUG   ] Got command RPT from MCS: ref #533779
2015-11-10 05:17:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533779  77 57336 19069019 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:17:49 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:17:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 05:17:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:17:57 [DEBUG   ] Finished updating board flags in 1.206 seconds
2015-11-10 05:17:58 [DEBUG   ] Got command DRX from MCS: ref #533795
2015-11-10 05:17:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:17:58 [DEBUG   ] run_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:17:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533795   8 57336 19078007 A  ERROR'
2015-11-10 05:17:58 [INFO    ] Finished starting DRX
2015-11-10 05:17:58 [DEBUG   ] Got command DRX from MCS: ref #533796
2015-11-10 05:17:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:17:58 [DEBUG   ] run_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:17:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533796   8 57336 19078227 A  ERROR'
2015-11-10 05:17:58 [DEBUG   ] Got command BAM from MCS: ref #533797
2015-11-10 05:17:58 [INFO    ] Finished processing BAM request
2015-11-10 05:17:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533797   8 57336 19078231 A  ERROR'
2015-11-10 05:17:58 [INFO    ] Finished starting DRX
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 05:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 05:17:58 [DEBUG   ] spawn_process: at 2015-11-10 05:17:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 05:18:09 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:18:29 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:18:49 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:18:57 [DEBUG   ] Finished updating board flags in 1.254 seconds
2015-11-10 05:19:09 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:19:29 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:19:49 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 05:19:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 05:19:57 [DEBUG   ] Finished updating board flags in 1.213 seconds
2015-11-10 05:19:58 [DEBUG   ] Got command DRX from MCS: ref #533801
2015-11-10 05:19:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:19:58 [DEBUG   ] run_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:19:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533801   8 57336 19198007 A  ERROR'
2015-11-10 05:19:58 [DEBUG   ] Got command DRX from MCS: ref #533800
2015-11-10 05:19:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:19:58 [DEBUG   ] run_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:19:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533800   8 57336 19198011 A  ERROR'
2015-11-10 05:19:58 [INFO    ] Finished starting DRX
2015-11-10 05:19:58 [INFO    ] Finished starting DRX
2015-11-10 05:19:58 [DEBUG   ] Got command DRX from MCS: ref #533802
2015-11-10 05:19:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:19:58 [DEBUG   ] run_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:19:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533802   8 57336 19198147 A  ERROR'
2015-11-10 05:19:58 [INFO    ] Finished starting DRX
2015-11-10 05:19:58 [DEBUG   ] Got command DRX from MCS: ref #533803
2015-11-10 05:19:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:19:58 [DEBUG   ] run_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:19:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533803   8 57336 19198283 A  ERROR'
2015-11-10 05:19:58 [INFO    ] Finished starting DRX
2015-11-10 05:19:58 [DEBUG   ] Got command BAM from MCS: ref #533805
2015-11-10 05:19:58 [INFO    ] Finished processing BAM request
2015-11-10 05:19:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533805   8 57336 19198419 A  ERROR'
2015-11-10 05:19:58 [DEBUG   ] Got command BAM from MCS: ref #533804
2015-11-10 05:19:58 [INFO    ] Finished processing BAM request
2015-11-10 05:19:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533804   8 57336 19198531 A  ERROR'
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 05:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 05:19:58 [DEBUG   ] spawn_process: at 2015-11-10 05:19:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 05:20:09 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:20:24 [DEBUG   ] Got command RPT from MCS: ref #533807
2015-11-10 05:20:24 [DEBUG   ] summary = ERROR
2015-11-10 05:20:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533807  13 57336 19224723 A  ERRORERROR'
2015-11-10 05:20:24 [DEBUG   ] Got command RPT from MCS: ref #533808
2015-11-10 05:20:24 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:20:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533808  77 57336 19224927 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:20:25 [DEBUG   ] Got command RPT from MCS: ref #533809
2015-11-10 05:20:25 [DEBUG   ] summary = ERROR
2015-11-10 05:20:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533809  13 57336 19225119 A  ERRORERROR'
2015-11-10 05:20:25 [DEBUG   ] Got command RPT from MCS: ref #533810
2015-11-10 05:20:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:20:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533810  77 57336 19225323 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:20:25 [DEBUG   ] Got command RPT from MCS: ref #533811
2015-11-10 05:20:25 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 05:20:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533811  12 57336 19225527 A  ERRORB‘™š'
2015-11-10 05:20:29 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:20:49 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:20:57 [DEBUG   ] Finished updating board flags in 1.123 seconds
2015-11-10 05:21:09 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:21:19 [DEBUG   ] Got command BAM from MCS: ref #533815
2015-11-10 05:21:19 [INFO    ] Finished processing BAM request
2015-11-10 05:21:19 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:21:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533815   8 57336 19279031 A  ERROR'
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 05:21:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 05:21:19 [DEBUG   ] spawn_process: at 2015-11-10 05:21:19 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 05:21:29 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:21:49 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:21:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-10 05:21:57 [DEBUG   ] Finished updating board flags in 1.019 seconds
2015-11-10 05:21:58 [DEBUG   ] Got command DRX from MCS: ref #533816
2015-11-10 05:21:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:21:58 [DEBUG   ] run_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:21:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533816   8 57336 19318007 A  ERROR'
2015-11-10 05:21:58 [INFO    ] Finished starting DRX
2015-11-10 05:21:58 [DEBUG   ] Got command DRX from MCS: ref #533817
2015-11-10 05:21:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:21:58 [DEBUG   ] run_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:21:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533817   8 57336 19318191 A  ERROR'
2015-11-10 05:21:58 [DEBUG   ] Got command BAM from MCS: ref #533818
2015-11-10 05:21:58 [INFO    ] Finished processing BAM request
2015-11-10 05:21:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533818   8 57336 19318195 A  ERROR'
2015-11-10 05:21:58 [INFO    ] Finished starting DRX
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 05:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 05:21:58 [DEBUG   ] spawn_process: at 2015-11-10 05:21:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 05:22:10 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:22:30 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:22:41 [DEBUG   ] Got command BAM from MCS: ref #533819
2015-11-10 05:22:41 [INFO    ] Finished processing BAM request
2015-11-10 05:22:41 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:22:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533819   8 57336 19361035 A  ERROR'
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 05:22:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 05:22:41 [DEBUG   ] spawn_process: at 2015-11-10 05:22:41 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 05:22:49 [DEBUG   ] Got command RPT from MCS: ref #533820
2015-11-10 05:22:49 [DEBUG   ] summary = ERROR
2015-11-10 05:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533820  13 57336 19369027 A  ERRORERROR'
2015-11-10 05:22:49 [DEBUG   ] Got command RPT from MCS: ref #533821
2015-11-10 05:22:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533821  77 57336 19369027 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:22:50 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2015-11-10 05:22:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:22:58 [DEBUG   ] Finished updating board flags in 1.987 seconds
2015-11-10 05:23:10 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:23:30 [DEBUG   ] Finished updating ant stats in 0.592 seconds
2015-11-10 05:23:50 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 05:23:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 05:23:58 [DEBUG   ] Got command DRX from MCS: ref #533840
2015-11-10 05:23:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:23:58 [DEBUG   ] run_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:23:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533840   8 57336 19438011 A  ERROR'
2015-11-10 05:23:58 [DEBUG   ] Got command DRX from MCS: ref #533841
2015-11-10 05:23:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:23:58 [DEBUG   ] run_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:23:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533841   8 57336 19438015 A  ERROR'
2015-11-10 05:23:58 [DEBUG   ] Got command DRX from MCS: ref #533843
2015-11-10 05:23:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:23:58 [DEBUG   ] run_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:23:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533843   8 57336 19438019 A  ERROR'
2015-11-10 05:23:58 [DEBUG   ] Got command DRX from MCS: ref #533842
2015-11-10 05:23:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:23:58 [DEBUG   ] run_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:23:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533842   8 57336 19438023 A  ERROR'
2015-11-10 05:23:58 [INFO    ] Finished starting DRX
2015-11-10 05:23:58 [INFO    ] Finished starting DRX
2015-11-10 05:23:58 [INFO    ] Finished starting DRX
2015-11-10 05:23:58 [INFO    ] Finished starting DRX
2015-11-10 05:23:58 [DEBUG   ] Got command BAM from MCS: ref #533845
2015-11-10 05:23:58 [INFO    ] Finished processing BAM request
2015-11-10 05:23:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533845   8 57336 19438159 A  ERROR'
2015-11-10 05:23:58 [DEBUG   ] Got command BAM from MCS: ref #533844
2015-11-10 05:23:58 [INFO    ] Finished processing BAM request
2015-11-10 05:23:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533844   8 57336 19438271 A  ERROR'
2015-11-10 05:23:58 [DEBUG   ] Finished updating board flags in 1.817 seconds
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 05:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 05:23:58 [DEBUG   ] spawn_process: at 2015-11-10 05:23:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 05:24:10 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:24:31 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:24:51 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:24:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:24:57 [DEBUG   ] Finished updating board flags in 1.227 seconds
2015-11-10 05:25:11 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2015-11-10 05:25:24 [DEBUG   ] Got command RPT from MCS: ref #533848
2015-11-10 05:25:24 [DEBUG   ] summary = ERROR
2015-11-10 05:25:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533848  13 57336 19524819 A  ERRORERROR'
2015-11-10 05:25:25 [DEBUG   ] Got command RPT from MCS: ref #533849
2015-11-10 05:25:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:25:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533849  77 57336 19525023 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:25:25 [DEBUG   ] Got command RPT from MCS: ref #533850
2015-11-10 05:25:25 [DEBUG   ] summary = ERROR
2015-11-10 05:25:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533850  13 57336 19525227 A  ERRORERROR'
2015-11-10 05:25:25 [DEBUG   ] Got command RPT from MCS: ref #533851
2015-11-10 05:25:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:25:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533851  77 57336 19525419 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:25:25 [DEBUG   ] Got command RPT from MCS: ref #533852
2015-11-10 05:25:25 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 05:25:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533852  12 57336 19525623 A  ERRORB•™š'
2015-11-10 05:25:31 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:25:51 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:25:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 05:25:57 [DEBUG   ] Finished updating board flags in 1.211 seconds
2015-11-10 05:25:58 [DEBUG   ] Got command DRX from MCS: ref #533858
2015-11-10 05:25:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:25:58 [DEBUG   ] run_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:25:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533858   8 57336 19558015 A  ERROR'
2015-11-10 05:25:58 [DEBUG   ] Got command DRX from MCS: ref #533859
2015-11-10 05:25:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:25:58 [DEBUG   ] run_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:25:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533859   8 57336 19558019 A  ERROR'
2015-11-10 05:25:58 [DEBUG   ] Got command DRX from MCS: ref #533861
2015-11-10 05:25:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:25:58 [DEBUG   ] run_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:25:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533861   8 57336 19558023 A  ERROR'
2015-11-10 05:25:58 [DEBUG   ] Got command DRX from MCS: ref #533860
2015-11-10 05:25:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:25:58 [DEBUG   ] run_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:25:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533860   8 57336 19558027 A  ERROR'
2015-11-10 05:25:58 [INFO    ] Finished starting DRX
2015-11-10 05:25:58 [INFO    ] Finished starting DRX
2015-11-10 05:25:58 [INFO    ] Finished starting DRX
2015-11-10 05:25:58 [INFO    ] Finished starting DRX
2015-11-10 05:25:58 [DEBUG   ] Got command BAM from MCS: ref #533863
2015-11-10 05:25:58 [INFO    ] Finished processing BAM request
2015-11-10 05:25:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533863   8 57336 19558175 A  ERROR'
2015-11-10 05:25:58 [DEBUG   ] Got command BAM from MCS: ref #533862
2015-11-10 05:25:58 [INFO    ] Finished processing BAM request
2015-11-10 05:25:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533862   8 57336 19558287 A  ERROR'
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 05:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 05:25:58 [DEBUG   ] spawn_process: at 2015-11-10 05:25:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 05:26:11 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:26:31 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:26:51 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:26:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 05:26:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 05:26:57 [DEBUG   ] Finished updating board flags in 1.211 seconds
2015-11-10 05:27:11 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:27:32 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:27:49 [DEBUG   ] Got command RPT from MCS: ref #533867
2015-11-10 05:27:49 [DEBUG   ] summary = ERROR
2015-11-10 05:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533867  13 57336 19669031 A  ERRORERROR'
2015-11-10 05:27:49 [DEBUG   ] Got command RPT from MCS: ref #533868
2015-11-10 05:27:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533868  77 57336 19669031 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:27:52 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:27:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:27:57 [DEBUG   ] Finished updating board flags in 1.173 seconds
2015-11-10 05:27:58 [DEBUG   ] Got command DRX from MCS: ref #533885
2015-11-10 05:27:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:27:58 [DEBUG   ] run_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:27:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533885   8 57336 19678007 A  ERROR'
2015-11-10 05:27:58 [DEBUG   ] Got command DRX from MCS: ref #533884
2015-11-10 05:27:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:27:58 [DEBUG   ] run_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:27:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533884   8 57336 19678011 A  ERROR'
2015-11-10 05:27:58 [INFO    ] Finished starting DRX
2015-11-10 05:27:58 [INFO    ] Finished starting DRX
2015-11-10 05:27:58 [DEBUG   ] Got command DRX from MCS: ref #533886
2015-11-10 05:27:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:27:58 [DEBUG   ] run_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:27:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533886   8 57336 19678099 A  ERROR'
2015-11-10 05:27:58 [INFO    ] Finished starting DRX
2015-11-10 05:27:58 [DEBUG   ] Got command DRX from MCS: ref #533887
2015-11-10 05:27:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:27:58 [DEBUG   ] run_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:27:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533887   8 57336 19678211 A  ERROR'
2015-11-10 05:27:58 [INFO    ] Finished starting DRX
2015-11-10 05:27:58 [DEBUG   ] Got command BAM from MCS: ref #533888
2015-11-10 05:27:58 [INFO    ] Finished processing BAM request
2015-11-10 05:27:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533888   8 57336 19678299 A  ERROR'
2015-11-10 05:27:58 [DEBUG   ] Got command BAM from MCS: ref #533889
2015-11-10 05:27:58 [INFO    ] Finished processing BAM request
2015-11-10 05:27:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533889   8 57336 19678399 A  ERROR'
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 05:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 05:27:58 [DEBUG   ] spawn_process: at 2015-11-10 05:27:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 05:28:12 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:28:32 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:28:52 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:28:57 [DEBUG   ] Finished updating board flags in 1.020 seconds
2015-11-10 05:29:12 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:29:32 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:29:52 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:29:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2015-11-10 05:29:57 [DEBUG   ] Finished updating board flags in 0.999 seconds
2015-11-10 05:29:58 [DEBUG   ] Got command DRX from MCS: ref #533893
2015-11-10 05:29:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:29:58 [DEBUG   ] run_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:29:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533893   8 57336 19798011 A  ERROR'
2015-11-10 05:29:58 [DEBUG   ] Got command DRX from MCS: ref #533894
2015-11-10 05:29:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:29:58 [DEBUG   ] run_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:29:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533894   8 57336 19798015 A  ERROR'
2015-11-10 05:29:58 [DEBUG   ] Got command DRX from MCS: ref #533895
2015-11-10 05:29:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:29:58 [DEBUG   ] run_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:29:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533895   8 57336 19798019 A  ERROR'
2015-11-10 05:29:58 [DEBUG   ] Got command DRX from MCS: ref #533896
2015-11-10 05:29:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:29:58 [DEBUG   ] run_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:29:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533896   8 57336 19798023 A  ERROR'
2015-11-10 05:29:58 [INFO    ] Finished starting DRX
2015-11-10 05:29:58 [INFO    ] Finished starting DRX
2015-11-10 05:29:58 [INFO    ] Finished starting DRX
2015-11-10 05:29:58 [INFO    ] Finished starting DRX
2015-11-10 05:29:58 [DEBUG   ] Got command BAM from MCS: ref #533897
2015-11-10 05:29:58 [INFO    ] Finished processing BAM request
2015-11-10 05:29:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533897   8 57336 19798175 A  ERROR'
2015-11-10 05:29:58 [DEBUG   ] Got command BAM from MCS: ref #533898
2015-11-10 05:29:58 [INFO    ] Finished processing BAM request
2015-11-10 05:29:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533898   8 57336 19798287 A  ERROR'
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 05:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 05:29:58 [DEBUG   ] spawn_process: at 2015-11-10 05:29:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 05:30:12 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2015-11-10 05:30:24 [DEBUG   ] Got command RPT from MCS: ref #533899
2015-11-10 05:30:24 [DEBUG   ] summary = ERROR
2015-11-10 05:30:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533899  13 57336 19824903 A  ERRORERROR'
2015-11-10 05:30:25 [DEBUG   ] Got command RPT from MCS: ref #533900
2015-11-10 05:30:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:30:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533900  77 57336 19825107 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:30:25 [DEBUG   ] Got command RPT from MCS: ref #533901
2015-11-10 05:30:25 [DEBUG   ] summary = ERROR
2015-11-10 05:30:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533901  13 57336 19825311 A  ERRORERROR'
2015-11-10 05:30:25 [DEBUG   ] Got command RPT from MCS: ref #533902
2015-11-10 05:30:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:30:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533902  77 57336 19825515 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:30:25 [DEBUG   ] Got command RPT from MCS: ref #533903
2015-11-10 05:30:25 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 05:30:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533903  12 57336 19825707 A  ERRORB•™š'
2015-11-10 05:30:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 05:30:52 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:30:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 05:30:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:30:58 [DEBUG   ] Finished updating board flags in 1.363 seconds
2015-11-10 05:31:12 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:31:32 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:31:52 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:31:58 [DEBUG   ] Got command DRX from MCS: ref #533904
2015-11-10 05:31:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:31:58 [DEBUG   ] run_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:31:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533904   8 57336 19918011 A  ERROR'
2015-11-10 05:31:58 [DEBUG   ] Got command DRX from MCS: ref #533905
2015-11-10 05:31:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:31:58 [DEBUG   ] run_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:31:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533905   8 57336 19918015 A  ERROR'
2015-11-10 05:31:58 [DEBUG   ] Got command DRX from MCS: ref #533906
2015-11-10 05:31:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:31:58 [DEBUG   ] run_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:31:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533906   8 57336 19918019 A  ERROR'
2015-11-10 05:31:58 [DEBUG   ] Got command DRX from MCS: ref #533907
2015-11-10 05:31:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:31:58 [DEBUG   ] run_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:31:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533907   8 57336 19918023 A  ERROR'
2015-11-10 05:31:58 [INFO    ] Finished starting DRX
2015-11-10 05:31:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 05:31:58 [INFO    ] Finished starting DRX
2015-11-10 05:31:58 [INFO    ] Finished starting DRX
2015-11-10 05:31:58 [INFO    ] Finished starting DRX
2015-11-10 05:31:58 [DEBUG   ] Got command BAM from MCS: ref #533908
2015-11-10 05:31:58 [INFO    ] Finished processing BAM request
2015-11-10 05:31:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533908   8 57336 19918207 A  ERROR'
2015-11-10 05:31:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 05:31:58 [DEBUG   ] Got command BAM from MCS: ref #533909
2015-11-10 05:31:58 [INFO    ] Finished processing BAM request
2015-11-10 05:31:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533909   8 57336 19918355 A  ERROR'
2015-11-10 05:31:58 [DEBUG   ] Finished updating board flags in 1.185 seconds
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 05:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 05:31:58 [DEBUG   ] spawn_process: at 2015-11-10 05:31:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 05:32:12 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:32:32 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:32:49 [DEBUG   ] Got command RPT from MCS: ref #533913
2015-11-10 05:32:49 [DEBUG   ] summary = ERROR
2015-11-10 05:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533913  13 57336 19969027 A  ERRORERROR'
2015-11-10 05:32:49 [DEBUG   ] Got command RPT from MCS: ref #533914
2015-11-10 05:32:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533914  77 57336 19969027 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:32:52 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:32:58 [DEBUG   ] Finished updating board flags in 1.077 seconds
2015-11-10 05:33:12 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:33:32 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:33:52 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:33:58 [DEBUG   ] Got command DRX from MCS: ref #533930
2015-11-10 05:33:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:33:58 [DEBUG   ] run_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:33:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533930   8 57336 20038011 A  ERROR'
2015-11-10 05:33:58 [DEBUG   ] Got command DRX from MCS: ref #533931
2015-11-10 05:33:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:33:58 [DEBUG   ] run_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:33:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533931   8 57336 20038015 A  ERROR'
2015-11-10 05:33:58 [DEBUG   ] Got command DRX from MCS: ref #533932
2015-11-10 05:33:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:33:58 [DEBUG   ] run_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:33:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533932   8 57336 20038019 A  ERROR'
2015-11-10 05:33:58 [DEBUG   ] Got command DRX from MCS: ref #533933
2015-11-10 05:33:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:33:58 [DEBUG   ] run_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:33:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533933   8 57336 20038023 A  ERROR'
2015-11-10 05:33:58 [INFO    ] Finished starting DRX
2015-11-10 05:33:58 [INFO    ] Finished starting DRX
2015-11-10 05:33:58 [INFO    ] Finished starting DRX
2015-11-10 05:33:58 [INFO    ] Finished starting DRX
2015-11-10 05:33:58 [DEBUG   ] Got command BAM from MCS: ref #533934
2015-11-10 05:33:58 [INFO    ] Finished processing BAM request
2015-11-10 05:33:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533934   8 57336 20038183 A  ERROR'
2015-11-10 05:33:58 [DEBUG   ] Got command BAM from MCS: ref #533935
2015-11-10 05:33:58 [INFO    ] Finished processing BAM request
2015-11-10 05:33:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533935   8 57336 20038283 A  ERROR'
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 05:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 05:33:58 [DEBUG   ] spawn_process: at 2015-11-10 05:33:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 05:33:59 [DEBUG   ] Finished updating board flags in 1.930 seconds
2015-11-10 05:34:12 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:34:32 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:34:52 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:34:58 [DEBUG   ] Finished updating board flags in 1.251 seconds
2015-11-10 05:35:12 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:35:25 [DEBUG   ] Got command RPT from MCS: ref #533937
2015-11-10 05:35:25 [DEBUG   ] summary = ERROR
2015-11-10 05:35:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533937  13 57336 20125031 A  ERRORERROR'
2015-11-10 05:35:25 [DEBUG   ] Got command RPT from MCS: ref #533938
2015-11-10 05:35:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:35:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533938  77 57336 20125235 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:35:25 [DEBUG   ] Got command RPT from MCS: ref #533939
2015-11-10 05:35:25 [DEBUG   ] summary = ERROR
2015-11-10 05:35:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533939  13 57336 20125439 A  ERRORERROR'
2015-11-10 05:35:25 [DEBUG   ] Got command RPT from MCS: ref #533940
2015-11-10 05:35:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:35:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533940  77 57336 20125631 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:35:25 [DEBUG   ] Got command RPT from MCS: ref #533941
2015-11-10 05:35:25 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 05:35:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533941  12 57336 20125835 A  ERRORB•™š'
2015-11-10 05:35:32 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:35:52 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:35:58 [DEBUG   ] Got command DRX from MCS: ref #533951
2015-11-10 05:35:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:35:58 [DEBUG   ] run_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:35:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533951   8 57336 20158015 A  ERROR'
2015-11-10 05:35:58 [DEBUG   ] Got command DRX from MCS: ref #533950
2015-11-10 05:35:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:35:58 [DEBUG   ] run_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:35:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533950   8 57336 20158019 A  ERROR'
2015-11-10 05:35:58 [DEBUG   ] Got command DRX from MCS: ref #533952
2015-11-10 05:35:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:35:58 [DEBUG   ] run_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:35:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533952   8 57336 20158023 A  ERROR'
2015-11-10 05:35:58 [DEBUG   ] Got command DRX from MCS: ref #533953
2015-11-10 05:35:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:35:58 [DEBUG   ] run_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:35:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533953   8 57336 20158027 A  ERROR'
2015-11-10 05:35:58 [INFO    ] Finished starting DRX
2015-11-10 05:35:58 [INFO    ] Finished starting DRX
2015-11-10 05:35:58 [INFO    ] Finished starting DRX
2015-11-10 05:35:58 [INFO    ] Finished starting DRX
2015-11-10 05:35:58 [DEBUG   ] Got command BAM from MCS: ref #533954
2015-11-10 05:35:58 [INFO    ] Finished processing BAM request
2015-11-10 05:35:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533954   8 57336 20158183 A  ERROR'
2015-11-10 05:35:58 [DEBUG   ] Got command BAM from MCS: ref #533955
2015-11-10 05:35:58 [INFO    ] Finished processing BAM request
2015-11-10 05:35:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533955   8 57336 20158295 A  ERROR'
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 05:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 05:35:58 [DEBUG   ] spawn_process: at 2015-11-10 05:35:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 05:35:59 [DEBUG   ] Finished updating board flags in 1.754 seconds
2015-11-10 05:36:12 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:36:32 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:36:52 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:36:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:36:58 [DEBUG   ] Finished updating board flags in 1.061 seconds
2015-11-10 05:37:12 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:37:32 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:37:49 [DEBUG   ] Got command RPT from MCS: ref #533956
2015-11-10 05:37:49 [DEBUG   ] summary = ERROR
2015-11-10 05:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533956  13 57336 20269023 A  ERRORERROR'
2015-11-10 05:37:49 [DEBUG   ] Got command RPT from MCS: ref #533957
2015-11-10 05:37:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533957  77 57336 20269023 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:37:52 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:37:58 [DEBUG   ] Got command DRX from MCS: ref #533973
2015-11-10 05:37:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:37:58 [DEBUG   ] run_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:37:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533973   8 57336 20278011 A  ERROR'
2015-11-10 05:37:58 [DEBUG   ] Got command DRX from MCS: ref #533974
2015-11-10 05:37:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:37:58 [DEBUG   ] run_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:37:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533974   8 57336 20278015 A  ERROR'
2015-11-10 05:37:58 [DEBUG   ] Got command DRX from MCS: ref #533975
2015-11-10 05:37:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:37:58 [DEBUG   ] run_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:37:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533975   8 57336 20278019 A  ERROR'
2015-11-10 05:37:58 [DEBUG   ] Got command DRX from MCS: ref #533976
2015-11-10 05:37:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:37:58 [DEBUG   ] run_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:37:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533976   8 57336 20278023 A  ERROR'
2015-11-10 05:37:58 [INFO    ] Finished starting DRX
2015-11-10 05:37:58 [INFO    ] Finished starting DRX
2015-11-10 05:37:58 [INFO    ] Finished starting DRX
2015-11-10 05:37:58 [INFO    ] Finished starting DRX
2015-11-10 05:37:58 [DEBUG   ] Got command BAM from MCS: ref #533977
2015-11-10 05:37:58 [INFO    ] Finished processing BAM request
2015-11-10 05:37:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533977   8 57336 20278219 A  ERROR'
2015-11-10 05:37:58 [DEBUG   ] Got command BAM from MCS: ref #533978
2015-11-10 05:37:58 [INFO    ] Finished processing BAM request
2015-11-10 05:37:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533978   8 57336 20278351 A  ERROR'
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 05:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 05:37:58 [DEBUG   ] spawn_process: at 2015-11-10 05:37:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 05:37:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:37:59 [DEBUG   ] Finished updating board flags in 1.546 seconds
2015-11-10 05:38:12 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:38:32 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:38:52 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:38:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2015-11-10 05:38:59 [DEBUG   ] Finished updating board flags in 1.406 seconds
2015-11-10 05:39:12 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:39:32 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:39:52 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 05:39:58 [DEBUG   ] Got command DRX from MCS: ref #533982
2015-11-10 05:39:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:39:58 [DEBUG   ] run_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533982   8 57336 20398011 A  ERROR'
2015-11-10 05:39:58 [DEBUG   ] Got command DRX from MCS: ref #533983
2015-11-10 05:39:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:39:58 [DEBUG   ] run_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533983   8 57336 20398015 A  ERROR'
2015-11-10 05:39:58 [DEBUG   ] Got command DRX from MCS: ref #533984
2015-11-10 05:39:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:39:58 [DEBUG   ] run_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533984   8 57336 20398019 A  ERROR'
2015-11-10 05:39:58 [DEBUG   ] Got command DRX from MCS: ref #533985
2015-11-10 05:39:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:39:58 [DEBUG   ] run_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533985   8 57336 20398023 A  ERROR'
2015-11-10 05:39:58 [INFO    ] Finished starting DRX
2015-11-10 05:39:58 [INFO    ] Finished starting DRX
2015-11-10 05:39:58 [INFO    ] Finished starting DRX
2015-11-10 05:39:58 [INFO    ] Finished starting DRX
2015-11-10 05:39:58 [DEBUG   ] Got command BAM from MCS: ref #533987
2015-11-10 05:39:58 [INFO    ] Finished processing BAM request
2015-11-10 05:39:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533987   8 57336 20398195 A  ERROR'
2015-11-10 05:39:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2015-11-10 05:39:58 [DEBUG   ] Got command BAM from MCS: ref #533986
2015-11-10 05:39:58 [INFO    ] Finished processing BAM request
2015-11-10 05:39:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   533986   8 57336 20398311 A  ERROR'
2015-11-10 05:39:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 05:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 05:39:58 [DEBUG   ] spawn_process: at 2015-11-10 05:39:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 05:39:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:39:59 [DEBUG   ] Finished updating board flags in 1.413 seconds
2015-11-10 05:40:12 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 05:40:25 [DEBUG   ] Got command RPT from MCS: ref #533988
2015-11-10 05:40:25 [DEBUG   ] summary = ERROR
2015-11-10 05:40:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533988  13 57336 20425143 A  ERRORERROR'
2015-11-10 05:40:25 [DEBUG   ] Got command RPT from MCS: ref #533989
2015-11-10 05:40:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:40:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533989  77 57336 20425335 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:40:25 [DEBUG   ] Got command RPT from MCS: ref #533990
2015-11-10 05:40:25 [DEBUG   ] summary = ERROR
2015-11-10 05:40:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533990  13 57336 20425539 A  ERRORERROR'
2015-11-10 05:40:25 [DEBUG   ] Got command RPT from MCS: ref #533991
2015-11-10 05:40:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:40:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533991  77 57336 20425743 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:40:25 [DEBUG   ] Got command RPT from MCS: ref #533992
2015-11-10 05:40:25 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 05:40:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   533992  12 57336 20425951 A  ERRORB–™š'
2015-11-10 05:40:32 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:40:52 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:40:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 05:40:59 [DEBUG   ] Finished updating board flags in 1.472 seconds
2015-11-10 05:41:12 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:41:32 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:41:52 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:41:58 [DEBUG   ] Got command DRX from MCS: ref #533997
2015-11-10 05:41:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:41:58 [DEBUG   ] run_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:41:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533997   8 57336 20518011 A  ERROR'
2015-11-10 05:41:58 [DEBUG   ] Got command DRX from MCS: ref #533996
2015-11-10 05:41:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:41:58 [DEBUG   ] run_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:41:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533996   8 57336 20518015 A  ERROR'
2015-11-10 05:41:58 [DEBUG   ] Got command DRX from MCS: ref #533999
2015-11-10 05:41:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:41:58 [DEBUG   ] run_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:41:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533999   8 57336 20518019 A  ERROR'
2015-11-10 05:41:58 [DEBUG   ] Got command DRX from MCS: ref #533998
2015-11-10 05:41:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:41:58 [DEBUG   ] run_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:41:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   533998   8 57336 20518023 A  ERROR'
2015-11-10 05:41:58 [INFO    ] Finished starting DRX
2015-11-10 05:41:58 [INFO    ] Finished starting DRX
2015-11-10 05:41:58 [INFO    ] Finished starting DRX
2015-11-10 05:41:58 [INFO    ] Finished starting DRX
2015-11-10 05:41:58 [DEBUG   ] Got command BAM from MCS: ref #534000
2015-11-10 05:41:58 [INFO    ] Finished processing BAM request
2015-11-10 05:41:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534000   8 57336 20518171 A  ERROR'
2015-11-10 05:41:58 [DEBUG   ] Got command BAM from MCS: ref #534001
2015-11-10 05:41:58 [INFO    ] Finished processing BAM request
2015-11-10 05:41:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534001   8 57336 20518295 A  ERROR'
2015-11-10 05:41:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 05:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 05:41:58 [DEBUG   ] spawn_process: at 2015-11-10 05:41:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 05:41:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 05:41:59 [DEBUG   ] Finished updating board flags in 1.314 seconds
2015-11-10 05:42:12 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2015-11-10 05:42:32 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:42:49 [DEBUG   ] Got command RPT from MCS: ref #534002
2015-11-10 05:42:49 [DEBUG   ] summary = ERROR
2015-11-10 05:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534002  13 57336 20569027 A  ERRORERROR'
2015-11-10 05:42:49 [DEBUG   ] Got command RPT from MCS: ref #534003
2015-11-10 05:42:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534003  77 57336 20569027 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:42:52 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:42:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:42:59 [DEBUG   ] Finished updating board flags in 1.209 seconds
2015-11-10 05:43:12 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:43:32 [DEBUG   ] Finished updating ant stats in 0.396 seconds
2015-11-10 05:43:53 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:43:58 [DEBUG   ] Got command DRX from MCS: ref #534019
2015-11-10 05:43:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:43:58 [DEBUG   ] run_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:43:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534019   8 57336 20638007 A  ERROR'
2015-11-10 05:43:58 [DEBUG   ] Got command DRX from MCS: ref #534020
2015-11-10 05:43:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:43:58 [DEBUG   ] run_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:43:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534020   8 57336 20638011 A  ERROR'
2015-11-10 05:43:58 [INFO    ] Finished starting DRX
2015-11-10 05:43:58 [INFO    ] Finished starting DRX
2015-11-10 05:43:58 [DEBUG   ] Got command DRX from MCS: ref #534021
2015-11-10 05:43:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:43:58 [DEBUG   ] run_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:43:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534021   8 57336 20638195 A  ERROR'
2015-11-10 05:43:58 [INFO    ] Finished starting DRX
2015-11-10 05:43:58 [DEBUG   ] Got command DRX from MCS: ref #534022
2015-11-10 05:43:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:43:58 [DEBUG   ] run_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:43:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534022   8 57336 20638331 A  ERROR'
2015-11-10 05:43:58 [INFO    ] Finished starting DRX
2015-11-10 05:43:58 [DEBUG   ] Got command BAM from MCS: ref #534024
2015-11-10 05:43:58 [INFO    ] Finished processing BAM request
2015-11-10 05:43:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534024   8 57336 20638491 A  ERROR'
2015-11-10 05:43:58 [DEBUG   ] Got command BAM from MCS: ref #534023
2015-11-10 05:43:58 [INFO    ] Finished processing BAM request
2015-11-10 05:43:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534023   8 57336 20638655 A  ERROR'
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 05:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 05:43:58 [DEBUG   ] spawn_process: at 2015-11-10 05:43:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 05:43:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 05:43:59 [DEBUG   ] Finished updating board flags in 1.206 seconds
2015-11-10 05:44:13 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2015-11-10 05:44:33 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:44:53 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 05:44:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 05:44:59 [DEBUG   ] Finished updating board flags in 1.187 seconds
2015-11-10 05:45:13 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2015-11-10 05:45:25 [DEBUG   ] Got command RPT from MCS: ref #534029
2015-11-10 05:45:25 [DEBUG   ] summary = ERROR
2015-11-10 05:45:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534029  13 57336 20725251 A  ERRORERROR'
2015-11-10 05:45:25 [DEBUG   ] Got command RPT from MCS: ref #534030
2015-11-10 05:45:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:45:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534030  77 57336 20725455 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:45:25 [DEBUG   ] Got command RPT from MCS: ref #534031
2015-11-10 05:45:25 [DEBUG   ] summary = ERROR
2015-11-10 05:45:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534031  13 57336 20725647 A  ERRORERROR'
2015-11-10 05:45:25 [DEBUG   ] Got command RPT from MCS: ref #534032
2015-11-10 05:45:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:45:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534032  77 57336 20725851 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:45:26 [DEBUG   ] Got command RPT from MCS: ref #534033
2015-11-10 05:45:26 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 05:45:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534033  12 57336 20726055 A  ERRORB–™š'
2015-11-10 05:45:33 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:45:53 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:45:58 [DEBUG   ] Got command DRX from MCS: ref #534040
2015-11-10 05:45:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:45:58 [DEBUG   ] run_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:45:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534040   8 57336 20758015 A  ERROR'
2015-11-10 05:45:58 [DEBUG   ] Got command DRX from MCS: ref #534039
2015-11-10 05:45:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:45:58 [DEBUG   ] run_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:45:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534039   8 57336 20758019 A  ERROR'
2015-11-10 05:45:58 [DEBUG   ] Got command DRX from MCS: ref #534041
2015-11-10 05:45:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:45:58 [DEBUG   ] run_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:45:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534041   8 57336 20758023 A  ERROR'
2015-11-10 05:45:58 [DEBUG   ] Got command DRX from MCS: ref #534042
2015-11-10 05:45:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:45:58 [DEBUG   ] run_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:45:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534042   8 57336 20758027 A  ERROR'
2015-11-10 05:45:58 [INFO    ] Finished starting DRX
2015-11-10 05:45:58 [INFO    ] Finished starting DRX
2015-11-10 05:45:58 [INFO    ] Finished starting DRX
2015-11-10 05:45:58 [INFO    ] Finished starting DRX
2015-11-10 05:45:58 [DEBUG   ] Got command BAM from MCS: ref #534044
2015-11-10 05:45:58 [INFO    ] Finished processing BAM request
2015-11-10 05:45:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534044   8 57336 20758247 A  ERROR'
2015-11-10 05:45:58 [DEBUG   ] Got command BAM from MCS: ref #534043
2015-11-10 05:45:58 [INFO    ] Finished processing BAM request
2015-11-10 05:45:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534043   8 57336 20758383 A  ERROR'
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 05:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 05:45:58 [DEBUG   ] spawn_process: at 2015-11-10 05:45:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 05:45:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:45:59 [DEBUG   ] Finished updating board flags in 1.029 seconds
2015-11-10 05:46:13 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:46:33 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 05:46:53 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:47:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 05:47:00 [DEBUG   ] Finished updating board flags in 1.986 seconds
2015-11-10 05:47:13 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:47:33 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2015-11-10 05:47:49 [DEBUG   ] Got command RPT from MCS: ref #534048
2015-11-10 05:47:49 [DEBUG   ] summary = ERROR
2015-11-10 05:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534048  13 57336 20869031 A  ERRORERROR'
2015-11-10 05:47:49 [DEBUG   ] Got command RPT from MCS: ref #534049
2015-11-10 05:47:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534049  77 57336 20869031 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:47:53 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2015-11-10 05:47:58 [DEBUG   ] Got command DRX from MCS: ref #534065
2015-11-10 05:47:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:47:58 [DEBUG   ] run_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:47:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534065   8 57336 20878007 A  ERROR'
2015-11-10 05:47:58 [DEBUG   ] Got command DRX from MCS: ref #534066
2015-11-10 05:47:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:47:58 [DEBUG   ] run_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:47:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534066   8 57336 20878011 A  ERROR'
2015-11-10 05:47:58 [INFO    ] Finished starting DRX
2015-11-10 05:47:58 [INFO    ] Finished starting DRX
2015-11-10 05:47:58 [DEBUG   ] Got command DRX from MCS: ref #534068
2015-11-10 05:47:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:47:58 [DEBUG   ] run_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:47:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534068   8 57336 20878183 A  ERROR'
2015-11-10 05:47:58 [INFO    ] Finished starting DRX
2015-11-10 05:47:58 [DEBUG   ] Got command DRX from MCS: ref #534067
2015-11-10 05:47:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:47:58 [DEBUG   ] run_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:47:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534067   8 57336 20878259 A  ERROR'
2015-11-10 05:47:58 [INFO    ] Finished starting DRX
2015-11-10 05:47:58 [DEBUG   ] Got command BAM from MCS: ref #534069
2015-11-10 05:47:58 [INFO    ] Finished processing BAM request
2015-11-10 05:47:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534069   8 57336 20878335 A  ERROR'
2015-11-10 05:47:58 [DEBUG   ] Got command BAM from MCS: ref #534070
2015-11-10 05:47:58 [INFO    ] Finished processing BAM request
2015-11-10 05:47:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534070   8 57336 20878451 A  ERROR'
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 05:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 05:47:58 [DEBUG   ] spawn_process: at 2015-11-10 05:47:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 05:47:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2015-11-10 05:48:00 [DEBUG   ] Finished updating board flags in 1.786 seconds
2015-11-10 05:48:13 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2015-11-10 05:48:33 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:48:54 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 05:48:59 [DEBUG   ] Finished updating board flags in 1.087 seconds
2015-11-10 05:49:14 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:49:34 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 05:49:54 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:49:58 [DEBUG   ] Got command DRX from MCS: ref #534071
2015-11-10 05:49:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:49:58 [DEBUG   ] run_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:49:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534071   8 57336 20998015 A  ERROR'
2015-11-10 05:49:58 [DEBUG   ] Got command DRX from MCS: ref #534072
2015-11-10 05:49:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:49:58 [DEBUG   ] run_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:49:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534072   8 57336 20998019 A  ERROR'
2015-11-10 05:49:58 [DEBUG   ] Got command DRX from MCS: ref #534073
2015-11-10 05:49:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:49:58 [DEBUG   ] run_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:49:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534073   8 57336 20998023 A  ERROR'
2015-11-10 05:49:58 [DEBUG   ] Got command DRX from MCS: ref #534074
2015-11-10 05:49:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:49:58 [DEBUG   ] run_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:49:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534074   8 57336 20998027 A  ERROR'
2015-11-10 05:49:58 [INFO    ] Finished starting DRX
2015-11-10 05:49:58 [INFO    ] Finished starting DRX
2015-11-10 05:49:58 [INFO    ] Finished starting DRX
2015-11-10 05:49:58 [INFO    ] Finished starting DRX
2015-11-10 05:49:58 [DEBUG   ] Got command BAM from MCS: ref #534075
2015-11-10 05:49:58 [INFO    ] Finished processing BAM request
2015-11-10 05:49:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534075   8 57336 20998179 A  ERROR'
2015-11-10 05:49:58 [DEBUG   ] Got command BAM from MCS: ref #534076
2015-11-10 05:49:58 [INFO    ] Finished processing BAM request
2015-11-10 05:49:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534076   8 57336 20998279 A  ERROR'
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 05:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 05:49:58 [DEBUG   ] spawn_process: at 2015-11-10 05:49:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 05:49:59 [DEBUG   ] Finished updating board flags in 0.962 seconds
2015-11-10 05:50:14 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:50:25 [DEBUG   ] Got command RPT from MCS: ref #534077
2015-11-10 05:50:25 [DEBUG   ] summary = ERROR
2015-11-10 05:50:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534077  13 57336 21025375 A  ERRORERROR'
2015-11-10 05:50:25 [DEBUG   ] Got command RPT from MCS: ref #534078
2015-11-10 05:50:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:50:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534078  77 57336 21025579 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:50:25 [DEBUG   ] Got command RPT from MCS: ref #534079
2015-11-10 05:50:25 [DEBUG   ] summary = ERROR
2015-11-10 05:50:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534079  13 57336 21025771 A  ERRORERROR'
2015-11-10 05:50:25 [DEBUG   ] Got command RPT from MCS: ref #534080
2015-11-10 05:50:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:50:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534080  77 57336 21025987 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:50:26 [DEBUG   ] Got command RPT from MCS: ref #534081
2015-11-10 05:50:26 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 05:50:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534081  12 57336 21026179 A  ERRORB–™š'
2015-11-10 05:50:34 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:50:54 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:51:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:51:00 [DEBUG   ] Finished updating board flags in 1.371 seconds
2015-11-10 05:51:14 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:51:34 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:51:55 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:51:58 [DEBUG   ] Got command DRX from MCS: ref #534086
2015-11-10 05:51:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:51:58 [DEBUG   ] run_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:51:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534086   8 57336 21118007 A  ERROR'
2015-11-10 05:51:58 [DEBUG   ] Got command DRX from MCS: ref #534085
2015-11-10 05:51:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:51:58 [DEBUG   ] run_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:51:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534085   8 57336 21118011 A  ERROR'
2015-11-10 05:51:58 [INFO    ] Finished starting DRX
2015-11-10 05:51:58 [INFO    ] Finished starting DRX
2015-11-10 05:51:58 [DEBUG   ] Got command DRX from MCS: ref #534087
2015-11-10 05:51:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:51:58 [DEBUG   ] run_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:51:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534087   8 57336 21118147 A  ERROR'
2015-11-10 05:51:58 [INFO    ] Finished starting DRX
2015-11-10 05:51:58 [DEBUG   ] Got command DRX from MCS: ref #534088
2015-11-10 05:51:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:51:58 [DEBUG   ] run_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:51:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534088   8 57336 21118247 A  ERROR'
2015-11-10 05:51:58 [INFO    ] Finished starting DRX
2015-11-10 05:51:58 [DEBUG   ] Got command BAM from MCS: ref #534089
2015-11-10 05:51:58 [INFO    ] Finished processing BAM request
2015-11-10 05:51:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534089   8 57336 21118371 A  ERROR'
2015-11-10 05:51:58 [DEBUG   ] Got command BAM from MCS: ref #534090
2015-11-10 05:51:58 [INFO    ] Finished processing BAM request
2015-11-10 05:51:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534090   8 57336 21118471 A  ERROR'
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 05:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 05:51:58 [DEBUG   ] spawn_process: at 2015-11-10 05:51:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 05:51:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 05:52:00 [DEBUG   ] Finished updating board flags in 1.192 seconds
2015-11-10 05:52:15 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:52:35 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:52:49 [DEBUG   ] Got command RPT from MCS: ref #534091
2015-11-10 05:52:49 [DEBUG   ] summary = ERROR
2015-11-10 05:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534091  13 57336 21169027 A  ERRORERROR'
2015-11-10 05:52:49 [DEBUG   ] Got command RPT from MCS: ref #534092
2015-11-10 05:52:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534092  77 57336 21169027 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:52:55 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:52:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 05:53:00 [DEBUG   ] Finished updating board flags in 0.992 seconds
2015-11-10 05:53:15 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:53:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 05:53:55 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:53:58 [DEBUG   ] Got command DRX from MCS: ref #534112
2015-11-10 05:53:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:53:58 [DEBUG   ] run_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:53:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534112   8 57336 21238015 A  ERROR'
2015-11-10 05:53:58 [DEBUG   ] Got command DRX from MCS: ref #534111
2015-11-10 05:53:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:53:58 [DEBUG   ] run_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:53:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534111   8 57336 21238019 A  ERROR'
2015-11-10 05:53:58 [DEBUG   ] Got command DRX from MCS: ref #534114
2015-11-10 05:53:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:53:58 [DEBUG   ] run_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:53:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534114   8 57336 21238023 A  ERROR'
2015-11-10 05:53:58 [DEBUG   ] Got command DRX from MCS: ref #534113
2015-11-10 05:53:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:53:58 [DEBUG   ] run_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:53:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534113   8 57336 21238027 A  ERROR'
2015-11-10 05:53:58 [INFO    ] Finished starting DRX
2015-11-10 05:53:58 [INFO    ] Finished starting DRX
2015-11-10 05:53:58 [INFO    ] Finished starting DRX
2015-11-10 05:53:58 [INFO    ] Finished starting DRX
2015-11-10 05:53:58 [DEBUG   ] Got command BAM from MCS: ref #534115
2015-11-10 05:53:58 [INFO    ] Finished processing BAM request
2015-11-10 05:53:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534115   8 57336 21238191 A  ERROR'
2015-11-10 05:53:58 [DEBUG   ] Got command BAM from MCS: ref #534116
2015-11-10 05:53:58 [INFO    ] Finished processing BAM request
2015-11-10 05:53:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534116   8 57336 21238291 A  ERROR'
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 05:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 05:53:58 [DEBUG   ] spawn_process: at 2015-11-10 05:53:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 05:54:00 [DEBUG   ] Finished updating board flags in 1.266 seconds
2015-11-10 05:54:15 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2015-11-10 05:54:35 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:54:55 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:55:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:55:00 [DEBUG   ] Finished updating board flags in 1.216 seconds
2015-11-10 05:55:16 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:55:25 [DEBUG   ] Got command RPT from MCS: ref #534118
2015-11-10 05:55:25 [DEBUG   ] summary = ERROR
2015-11-10 05:55:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534118  13 57336 21325471 A  ERRORERROR'
2015-11-10 05:55:25 [DEBUG   ] Got command RPT from MCS: ref #534119
2015-11-10 05:55:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:55:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534119  77 57336 21325675 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:55:25 [DEBUG   ] Got command RPT from MCS: ref #534120
2015-11-10 05:55:25 [DEBUG   ] summary = ERROR
2015-11-10 05:55:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534120  13 57336 21325867 A  ERRORERROR'
2015-11-10 05:55:26 [DEBUG   ] Got command RPT from MCS: ref #534121
2015-11-10 05:55:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:55:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534121  77 57336 21326071 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:55:26 [DEBUG   ] Got command RPT from MCS: ref #534122
2015-11-10 05:55:26 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 05:55:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534122  12 57336 21326275 A  ERRORB•™š'
2015-11-10 05:55:36 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:55:56 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 05:55:58 [DEBUG   ] Got command DRX from MCS: ref #534128
2015-11-10 05:55:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:55:58 [DEBUG   ] run_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:55:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534128   8 57336 21358015 A  ERROR'
2015-11-10 05:55:58 [DEBUG   ] Got command DRX from MCS: ref #534129
2015-11-10 05:55:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:55:58 [DEBUG   ] run_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:55:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534129   8 57336 21358019 A  ERROR'
2015-11-10 05:55:58 [DEBUG   ] Got command DRX from MCS: ref #534130
2015-11-10 05:55:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:55:58 [DEBUG   ] run_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:55:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534130   8 57336 21358023 A  ERROR'
2015-11-10 05:55:58 [DEBUG   ] Got command DRX from MCS: ref #534131
2015-11-10 05:55:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:55:58 [DEBUG   ] run_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:55:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534131   8 57336 21358027 A  ERROR'
2015-11-10 05:55:58 [INFO    ] Finished starting DRX
2015-11-10 05:55:58 [INFO    ] Finished starting DRX
2015-11-10 05:55:58 [INFO    ] Finished starting DRX
2015-11-10 05:55:58 [INFO    ] Finished starting DRX
2015-11-10 05:55:58 [DEBUG   ] Got command BAM from MCS: ref #534132
2015-11-10 05:55:58 [INFO    ] Finished processing BAM request
2015-11-10 05:55:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534132   8 57336 21358215 A  ERROR'
2015-11-10 05:55:58 [DEBUG   ] Got command BAM from MCS: ref #534133
2015-11-10 05:55:58 [INFO    ] Finished processing BAM request
2015-11-10 05:55:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534133   8 57336 21358327 A  ERROR'
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 05:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 05:55:58 [DEBUG   ] spawn_process: at 2015-11-10 05:55:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 05:56:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:56:00 [DEBUG   ] Finished updating board flags in 1.162 seconds
2015-11-10 05:56:16 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2015-11-10 05:56:36 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 05:56:56 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:57:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 05:57:00 [DEBUG   ] Finished updating board flags in 1.038 seconds
2015-11-10 05:57:16 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:57:36 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:57:49 [DEBUG   ] Got command RPT from MCS: ref #534137
2015-11-10 05:57:49 [DEBUG   ] summary = ERROR
2015-11-10 05:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534137  13 57336 21469031 A  ERRORERROR'
2015-11-10 05:57:49 [DEBUG   ] Got command RPT from MCS: ref #534138
2015-11-10 05:57:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 05:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534138  77 57336 21469031 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 05:57:56 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 05:57:58 [DEBUG   ] Got command DRX from MCS: ref #534155
2015-11-10 05:57:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:57:58 [DEBUG   ] run_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:57:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534155   8 57336 21478011 A  ERROR'
2015-11-10 05:57:58 [DEBUG   ] Got command DRX from MCS: ref #534154
2015-11-10 05:57:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:57:58 [DEBUG   ] run_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:57:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534154   8 57336 21478015 A  ERROR'
2015-11-10 05:57:58 [DEBUG   ] Got command DRX from MCS: ref #534156
2015-11-10 05:57:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:57:58 [DEBUG   ] run_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:57:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534156   8 57336 21478019 A  ERROR'
2015-11-10 05:57:58 [DEBUG   ] Got command DRX from MCS: ref #534157
2015-11-10 05:57:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:57:58 [DEBUG   ] run_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:57:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534157   8 57336 21478023 A  ERROR'
2015-11-10 05:57:58 [INFO    ] Finished starting DRX
2015-11-10 05:57:58 [INFO    ] Finished starting DRX
2015-11-10 05:57:58 [INFO    ] Finished starting DRX
2015-11-10 05:57:58 [INFO    ] Finished starting DRX
2015-11-10 05:57:58 [DEBUG   ] Got command BAM from MCS: ref #534158
2015-11-10 05:57:58 [INFO    ] Finished processing BAM request
2015-11-10 05:57:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534158   8 57336 21478199 A  ERROR'
2015-11-10 05:57:58 [DEBUG   ] Got command BAM from MCS: ref #534159
2015-11-10 05:57:58 [INFO    ] Finished processing BAM request
2015-11-10 05:57:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534159   8 57336 21478299 A  ERROR'
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 05:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 05:57:58 [DEBUG   ] spawn_process: at 2015-11-10 05:57:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 05:58:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 05:58:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 05:58:00 [DEBUG   ] Finished updating board flags in 0.968 seconds
2015-11-10 05:58:16 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 05:58:36 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:58:56 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 05:59:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 05:59:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 05:59:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 05:59:01 [DEBUG   ] Finished updating board flags in 1.358 seconds
2015-11-10 05:59:17 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 05:59:37 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:59:57 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 05:59:58 [DEBUG   ] Got command DRX from MCS: ref #534163
2015-11-10 05:59:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:59:58 [DEBUG   ] run_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 05:59:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534163   8 57336 21598015 A  ERROR'
2015-11-10 05:59:58 [DEBUG   ] Got command DRX from MCS: ref #534164
2015-11-10 05:59:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:59:58 [DEBUG   ] run_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 05:59:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534164   8 57336 21598019 A  ERROR'
2015-11-10 05:59:58 [DEBUG   ] Got command DRX from MCS: ref #534165
2015-11-10 05:59:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:59:58 [DEBUG   ] run_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 05:59:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534165   8 57336 21598023 A  ERROR'
2015-11-10 05:59:58 [DEBUG   ] Got command DRX from MCS: ref #534166
2015-11-10 05:59:58 [INFO    ] sending drx command to lwa16
2015-11-10 05:59:58 [DEBUG   ] run_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 05:59:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 05:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534166   8 57336 21598027 A  ERROR'
2015-11-10 05:59:58 [INFO    ] Finished starting DRX
2015-11-10 05:59:58 [INFO    ] Finished starting DRX
2015-11-10 05:59:58 [INFO    ] Finished starting DRX
2015-11-10 05:59:58 [INFO    ] Finished starting DRX
2015-11-10 05:59:58 [DEBUG   ] Got command BAM from MCS: ref #534167
2015-11-10 05:59:58 [INFO    ] Finished processing BAM request
2015-11-10 05:59:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534167   8 57336 21598187 A  ERROR'
2015-11-10 05:59:58 [DEBUG   ] Got command BAM from MCS: ref #534168
2015-11-10 05:59:58 [INFO    ] Finished processing BAM request
2015-11-10 05:59:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 05:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534168   8 57336 21598335 A  ERROR'
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 05:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 05:59:58 [DEBUG   ] spawn_process: at 2015-11-10 05:59:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 06:00:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2015-11-10 06:00:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 06:00:01 [DEBUG   ] Finished updating board flags in 1.191 seconds
2015-11-10 06:00:17 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:00:25 [DEBUG   ] Got command RPT from MCS: ref #534169
2015-11-10 06:00:25 [DEBUG   ] summary = ERROR
2015-11-10 06:00:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534169  13 57336 21625563 A  ERRORERROR'
2015-11-10 06:00:25 [DEBUG   ] Got command RPT from MCS: ref #534170
2015-11-10 06:00:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:00:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534170  77 57336 21625767 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:00:25 [DEBUG   ] Got command RPT from MCS: ref #534171
2015-11-10 06:00:25 [DEBUG   ] summary = ERROR
2015-11-10 06:00:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534171  13 57336 21625959 A  ERRORERROR'
2015-11-10 06:00:26 [DEBUG   ] Got command RPT from MCS: ref #534172
2015-11-10 06:00:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:00:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534172  77 57336 21626163 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:00:26 [DEBUG   ] Got command RPT from MCS: ref #534173
2015-11-10 06:00:26 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 06:00:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534173  12 57336 21626367 A  ERRORB–™š'
2015-11-10 06:00:37 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:00:57 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:01:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 06:01:01 [DEBUG   ] Finished updating board flags in 1.014 seconds
2015-11-10 06:01:17 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:01:37 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:01:57 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:01:58 [DEBUG   ] Got command DRX from MCS: ref #534174
2015-11-10 06:01:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:01:58 [DEBUG   ] run_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:01:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534174   8 57336 21718011 A  ERROR'
2015-11-10 06:01:58 [DEBUG   ] Got command DRX from MCS: ref #534175
2015-11-10 06:01:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:01:58 [DEBUG   ] run_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:01:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534175   8 57336 21718015 A  ERROR'
2015-11-10 06:01:58 [DEBUG   ] Got command DRX from MCS: ref #534176
2015-11-10 06:01:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:01:58 [DEBUG   ] run_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:01:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534176   8 57336 21718019 A  ERROR'
2015-11-10 06:01:58 [DEBUG   ] Got command DRX from MCS: ref #534177
2015-11-10 06:01:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:01:58 [DEBUG   ] run_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:01:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534177   8 57336 21718023 A  ERROR'
2015-11-10 06:01:58 [INFO    ] Finished starting DRX
2015-11-10 06:01:58 [INFO    ] Finished starting DRX
2015-11-10 06:01:58 [INFO    ] Finished starting DRX
2015-11-10 06:01:58 [INFO    ] Finished starting DRX
2015-11-10 06:01:58 [DEBUG   ] Got command BAM from MCS: ref #534178
2015-11-10 06:01:58 [INFO    ] Finished processing BAM request
2015-11-10 06:01:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534178   8 57336 21718187 A  ERROR'
2015-11-10 06:01:58 [DEBUG   ] Got command BAM from MCS: ref #534179
2015-11-10 06:01:58 [INFO    ] Finished processing BAM request
2015-11-10 06:01:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534179   8 57336 21718287 A  ERROR'
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 06:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 06:01:58 [DEBUG   ] spawn_process: at 2015-11-10 06:01:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 06:02:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 06:02:01 [DEBUG   ] Finished updating board flags in 1.008 seconds
2015-11-10 06:02:17 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 06:02:37 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:02:49 [DEBUG   ] Got command RPT from MCS: ref #534183
2015-11-10 06:02:49 [DEBUG   ] summary = ERROR
2015-11-10 06:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534183  13 57336 21769027 A  ERRORERROR'
2015-11-10 06:02:49 [DEBUG   ] Got command RPT from MCS: ref #534184
2015-11-10 06:02:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534184  77 57336 21769027 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:02:57 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 06:03:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 06:03:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 06:03:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:03:01 [DEBUG   ] Finished updating board flags in 0.987 seconds
2015-11-10 06:03:17 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:03:37 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:03:57 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:03:58 [DEBUG   ] Got command DRX from MCS: ref #534201
2015-11-10 06:03:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:03:58 [DEBUG   ] run_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:03:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534201   8 57336 21838015 A  ERROR'
2015-11-10 06:03:58 [DEBUG   ] Got command DRX from MCS: ref #534200
2015-11-10 06:03:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:03:58 [DEBUG   ] run_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:03:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534200   8 57336 21838019 A  ERROR'
2015-11-10 06:03:58 [DEBUG   ] Got command DRX from MCS: ref #534202
2015-11-10 06:03:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:03:58 [DEBUG   ] run_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:03:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534202   8 57336 21838023 A  ERROR'
2015-11-10 06:03:58 [DEBUG   ] Got command DRX from MCS: ref #534203
2015-11-10 06:03:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:03:58 [DEBUG   ] run_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:03:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534203   8 57336 21838027 A  ERROR'
2015-11-10 06:03:58 [INFO    ] Finished starting DRX
2015-11-10 06:03:58 [INFO    ] Finished starting DRX
2015-11-10 06:03:58 [INFO    ] Finished starting DRX
2015-11-10 06:03:58 [INFO    ] Finished starting DRX
2015-11-10 06:03:58 [DEBUG   ] Got command BAM from MCS: ref #534205
2015-11-10 06:03:58 [INFO    ] Finished processing BAM request
2015-11-10 06:03:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534205   8 57336 21838203 A  ERROR'
2015-11-10 06:03:58 [DEBUG   ] Got command BAM from MCS: ref #534204
2015-11-10 06:03:58 [INFO    ] Finished processing BAM request
2015-11-10 06:03:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534204   8 57336 21838315 A  ERROR'
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 06:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 06:03:58 [DEBUG   ] spawn_process: at 2015-11-10 06:03:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 06:04:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 06:04:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:04:01 [DEBUG   ] Finished updating board flags in 1.277 seconds
2015-11-10 06:04:17 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:04:37 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:04:57 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:05:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 06:05:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:05:02 [DEBUG   ] Finished updating board flags in 1.743 seconds
2015-11-10 06:05:17 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:05:25 [DEBUG   ] Got command RPT from MCS: ref #534207
2015-11-10 06:05:25 [DEBUG   ] summary = ERROR
2015-11-10 06:05:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534207  13 57336 21925643 A  ERRORERROR'
2015-11-10 06:05:25 [DEBUG   ] Got command RPT from MCS: ref #534208
2015-11-10 06:05:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:05:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534208  77 57336 21925847 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:05:26 [DEBUG   ] Got command RPT from MCS: ref #534209
2015-11-10 06:05:26 [DEBUG   ] summary = ERROR
2015-11-10 06:05:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534209  13 57336 21926051 A  ERRORERROR'
2015-11-10 06:05:26 [DEBUG   ] Got command RPT from MCS: ref #534210
2015-11-10 06:05:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:05:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534210  77 57336 21926243 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:05:26 [DEBUG   ] Got command RPT from MCS: ref #534211
2015-11-10 06:05:26 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 06:05:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534211  12 57336 21926447 A  ERRORB•™š'
2015-11-10 06:05:37 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:05:57 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2015-11-10 06:05:58 [DEBUG   ] Got command DRX from MCS: ref #534221
2015-11-10 06:05:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:05:58 [DEBUG   ] run_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:05:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534221   8 57336 21958007 A  ERROR'
2015-11-10 06:05:58 [DEBUG   ] Got command DRX from MCS: ref #534220
2015-11-10 06:05:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:05:58 [DEBUG   ] run_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:05:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534220   8 57336 21958011 A  ERROR'
2015-11-10 06:05:58 [INFO    ] Finished starting DRX
2015-11-10 06:05:58 [INFO    ] Finished starting DRX
2015-11-10 06:05:58 [DEBUG   ] Got command DRX from MCS: ref #534223
2015-11-10 06:05:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:05:58 [DEBUG   ] run_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:05:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534223   8 57336 21958171 A  ERROR'
2015-11-10 06:05:58 [INFO    ] Finished starting DRX
2015-11-10 06:05:58 [DEBUG   ] Got command DRX from MCS: ref #534222
2015-11-10 06:05:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:05:58 [DEBUG   ] run_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:05:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534222   8 57336 21958307 A  ERROR'
2015-11-10 06:05:58 [INFO    ] Finished starting DRX
2015-11-10 06:05:58 [DEBUG   ] Got command BAM from MCS: ref #534225
2015-11-10 06:05:58 [INFO    ] Finished processing BAM request
2015-11-10 06:05:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534225   8 57336 21958443 A  ERROR'
2015-11-10 06:05:58 [DEBUG   ] Got command BAM from MCS: ref #534224
2015-11-10 06:05:58 [INFO    ] Finished processing BAM request
2015-11-10 06:05:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534224   8 57336 21958615 A  ERROR'
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 06:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 06:05:58 [DEBUG   ] spawn_process: at 2015-11-10 06:05:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 06:06:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 06:06:01 [DEBUG   ] Finished updating board flags in 1.033 seconds
2015-11-10 06:06:17 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:06:38 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 06:06:57 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:07:02 [DEBUG   ] Finished updating board flags in 1.633 seconds
2015-11-10 06:07:17 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:07:37 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:07:49 [DEBUG   ] Got command RPT from MCS: ref #534226
2015-11-10 06:07:49 [DEBUG   ] summary = ERROR
2015-11-10 06:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534226  13 57336 22069039 A  ERRORERROR'
2015-11-10 06:07:49 [DEBUG   ] Got command RPT from MCS: ref #534227
2015-11-10 06:07:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534227  77 57336 22069039 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:07:57 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:07:58 [DEBUG   ] Got command DRX from MCS: ref #534243
2015-11-10 06:07:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:07:58 [DEBUG   ] run_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:07:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534243   8 57336 22078007 A  ERROR'
2015-11-10 06:07:58 [DEBUG   ] Got command DRX from MCS: ref #534244
2015-11-10 06:07:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:07:58 [DEBUG   ] run_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:07:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534244   8 57336 22078011 A  ERROR'
2015-11-10 06:07:58 [INFO    ] Finished starting DRX
2015-11-10 06:07:58 [INFO    ] Finished starting DRX
2015-11-10 06:07:58 [DEBUG   ] Got command DRX from MCS: ref #534245
2015-11-10 06:07:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:07:58 [DEBUG   ] run_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:07:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534245   8 57336 22078171 A  ERROR'
2015-11-10 06:07:58 [INFO    ] Finished starting DRX
2015-11-10 06:07:58 [DEBUG   ] Got command DRX from MCS: ref #534246
2015-11-10 06:07:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:07:58 [DEBUG   ] run_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:07:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534246   8 57336 22078295 A  ERROR'
2015-11-10 06:07:58 [INFO    ] Finished starting DRX
2015-11-10 06:07:58 [DEBUG   ] Got command BAM from MCS: ref #534248
2015-11-10 06:07:58 [INFO    ] Finished processing BAM request
2015-11-10 06:07:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534248   8 57336 22078443 A  ERROR'
2015-11-10 06:07:58 [DEBUG   ] Got command BAM from MCS: ref #534247
2015-11-10 06:07:58 [INFO    ] Finished processing BAM request
2015-11-10 06:07:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534247   8 57336 22078579 A  ERROR'
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 06:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 06:07:58 [DEBUG   ] spawn_process: at 2015-11-10 06:07:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 06:08:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 06:08:01 [DEBUG   ] Finished updating board flags in 0.973 seconds
2015-11-10 06:08:17 [DEBUG   ] Finished updating ant stats in 0.396 seconds
2015-11-10 06:08:38 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:08:58 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 06:09:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:09:02 [DEBUG   ] Finished updating board flags in 1.360 seconds
2015-11-10 06:09:18 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2015-11-10 06:09:38 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:09:58 [DEBUG   ] Got command DRX from MCS: ref #534252
2015-11-10 06:09:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:09:58 [DEBUG   ] run_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:09:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534252   8 57336 22198015 A  ERROR'
2015-11-10 06:09:58 [DEBUG   ] Got command DRX from MCS: ref #534253
2015-11-10 06:09:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:09:58 [DEBUG   ] run_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:09:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534253   8 57336 22198019 A  ERROR'
2015-11-10 06:09:58 [DEBUG   ] Got command DRX from MCS: ref #534254
2015-11-10 06:09:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:09:58 [DEBUG   ] run_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:09:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534254   8 57336 22198023 A  ERROR'
2015-11-10 06:09:58 [DEBUG   ] Got command DRX from MCS: ref #534255
2015-11-10 06:09:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:09:58 [DEBUG   ] run_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:09:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534255   8 57336 22198027 A  ERROR'
2015-11-10 06:09:58 [INFO    ] Finished starting DRX
2015-11-10 06:09:58 [INFO    ] Finished starting DRX
2015-11-10 06:09:58 [INFO    ] Finished starting DRX
2015-11-10 06:09:58 [INFO    ] Finished starting DRX
2015-11-10 06:09:58 [DEBUG   ] Got command BAM from MCS: ref #534256
2015-11-10 06:09:58 [INFO    ] Finished processing BAM request
2015-11-10 06:09:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534256   8 57336 22198191 A  ERROR'
2015-11-10 06:09:58 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:09:58 [DEBUG   ] Got command BAM from MCS: ref #534257
2015-11-10 06:09:58 [INFO    ] Finished processing BAM request
2015-11-10 06:09:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534257   8 57336 22198315 A  ERROR'
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 06:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 06:09:58 [DEBUG   ] spawn_process: at 2015-11-10 06:09:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 06:10:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 06:10:02 [DEBUG   ] Finished updating board flags in 1.194 seconds
2015-11-10 06:10:18 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 06:10:25 [DEBUG   ] Got command RPT from MCS: ref #534258
2015-11-10 06:10:25 [DEBUG   ] summary = ERROR
2015-11-10 06:10:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534258  13 57336 22225739 A  ERRORERROR'
2015-11-10 06:10:25 [DEBUG   ] Got command RPT from MCS: ref #534259
2015-11-10 06:10:25 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:10:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534259  77 57336 22225943 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:10:26 [DEBUG   ] Got command RPT from MCS: ref #534260
2015-11-10 06:10:26 [DEBUG   ] summary = ERROR
2015-11-10 06:10:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534260  13 57336 22226135 A  ERRORERROR'
2015-11-10 06:10:26 [DEBUG   ] Got command RPT from MCS: ref #534261
2015-11-10 06:10:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:10:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534261  77 57336 22226339 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:10:26 [DEBUG   ] Got command RPT from MCS: ref #534262
2015-11-10 06:10:26 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 06:10:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534262  12 57336 22226543 A  ERRORB–™š'
2015-11-10 06:10:38 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 06:10:58 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:11:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 06:11:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:11:02 [DEBUG   ] Finished updating board flags in 1.001 seconds
2015-11-10 06:11:18 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:11:38 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:11:58 [DEBUG   ] Got command DRX from MCS: ref #534266
2015-11-10 06:11:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:11:58 [DEBUG   ] run_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:11:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534266   8 57336 22318011 A  ERROR'
2015-11-10 06:11:58 [DEBUG   ] Got command DRX from MCS: ref #534267
2015-11-10 06:11:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:11:58 [DEBUG   ] run_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:11:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534267   8 57336 22318015 A  ERROR'
2015-11-10 06:11:58 [DEBUG   ] Got command DRX from MCS: ref #534268
2015-11-10 06:11:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:11:58 [DEBUG   ] run_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:11:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534268   8 57336 22318019 A  ERROR'
2015-11-10 06:11:58 [DEBUG   ] Got command DRX from MCS: ref #534269
2015-11-10 06:11:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:11:58 [DEBUG   ] run_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:11:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534269   8 57336 22318023 A  ERROR'
2015-11-10 06:11:58 [INFO    ] Finished starting DRX
2015-11-10 06:11:58 [INFO    ] Finished starting DRX
2015-11-10 06:11:58 [INFO    ] Finished starting DRX
2015-11-10 06:11:58 [INFO    ] Finished starting DRX
2015-11-10 06:11:58 [DEBUG   ] Got command BAM from MCS: ref #534270
2015-11-10 06:11:58 [INFO    ] Finished processing BAM request
2015-11-10 06:11:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534270   8 57336 22318147 A  ERROR'
2015-11-10 06:11:58 [DEBUG   ] Got command BAM from MCS: ref #534271
2015-11-10 06:11:58 [INFO    ] Finished processing BAM request
2015-11-10 06:11:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534271   8 57336 22318223 A  ERROR'
2015-11-10 06:11:58 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 06:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 06:11:58 [DEBUG   ] spawn_process: at 2015-11-10 06:11:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 06:12:02 [DEBUG   ] Finished updating board flags in 1.470 seconds
2015-11-10 06:12:18 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:12:38 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:12:49 [DEBUG   ] Got command RPT from MCS: ref #534272
2015-11-10 06:12:49 [DEBUG   ] summary = ERROR
2015-11-10 06:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534272  13 57336 22369031 A  ERRORERROR'
2015-11-10 06:12:49 [DEBUG   ] Got command RPT from MCS: ref #534273
2015-11-10 06:12:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534273  77 57336 22369031 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:12:58 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:13:02 [DEBUG   ] Finished updating board flags in 1.377 seconds
2015-11-10 06:13:19 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:13:39 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:13:58 [DEBUG   ] Got command DRX from MCS: ref #534289
2015-11-10 06:13:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:13:58 [DEBUG   ] run_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:13:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534289   8 57336 22438007 A  ERROR'
2015-11-10 06:13:58 [DEBUG   ] Got command DRX from MCS: ref #534290
2015-11-10 06:13:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:13:58 [DEBUG   ] run_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:13:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534290   8 57336 22438011 A  ERROR'
2015-11-10 06:13:58 [INFO    ] Finished starting DRX
2015-11-10 06:13:58 [INFO    ] Finished starting DRX
2015-11-10 06:13:58 [DEBUG   ] Got command DRX from MCS: ref #534291
2015-11-10 06:13:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:13:58 [DEBUG   ] run_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:13:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534291   8 57336 22438163 A  ERROR'
2015-11-10 06:13:58 [INFO    ] Finished starting DRX
2015-11-10 06:13:58 [DEBUG   ] Got command DRX from MCS: ref #534292
2015-11-10 06:13:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:13:58 [DEBUG   ] run_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:13:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534292   8 57336 22438263 A  ERROR'
2015-11-10 06:13:58 [INFO    ] Finished starting DRX
2015-11-10 06:13:58 [DEBUG   ] Got command BAM from MCS: ref #534294
2015-11-10 06:13:58 [INFO    ] Finished processing BAM request
2015-11-10 06:13:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534294   8 57336 22438351 A  ERROR'
2015-11-10 06:13:58 [DEBUG   ] Got command BAM from MCS: ref #534293
2015-11-10 06:13:58 [INFO    ] Finished processing BAM request
2015-11-10 06:13:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534293   8 57336 22438451 A  ERROR'
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 06:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 06:13:58 [DEBUG   ] spawn_process: at 2015-11-10 06:13:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 06:13:59 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:14:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 06:14:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:14:02 [DEBUG   ] Finished updating board flags in 1.223 seconds
2015-11-10 06:14:19 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:14:39 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:14:59 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:15:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 06:15:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:15:02 [DEBUG   ] Finished updating board flags in 1.191 seconds
2015-11-10 06:15:19 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:15:25 [DEBUG   ] Got command RPT from MCS: ref #534299
2015-11-10 06:15:25 [DEBUG   ] summary = ERROR
2015-11-10 06:15:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534299  13 57336 22525823 A  ERRORERROR'
2015-11-10 06:15:26 [DEBUG   ] Got command RPT from MCS: ref #534300
2015-11-10 06:15:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:15:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534300  77 57336 22526027 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:15:26 [DEBUG   ] Got command RPT from MCS: ref #534301
2015-11-10 06:15:26 [DEBUG   ] summary = ERROR
2015-11-10 06:15:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534301  13 57336 22526231 A  ERRORERROR'
2015-11-10 06:15:26 [DEBUG   ] Got command RPT from MCS: ref #534302
2015-11-10 06:15:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:15:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534302  77 57336 22526423 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:15:26 [DEBUG   ] Got command RPT from MCS: ref #534303
2015-11-10 06:15:26 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 06:15:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534303  12 57336 22526627 A  ERRORB–™š'
2015-11-10 06:15:39 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:15:58 [DEBUG   ] Got command DRX from MCS: ref #534310
2015-11-10 06:15:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:15:58 [DEBUG   ] run_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:15:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534310   8 57336 22558015 A  ERROR'
2015-11-10 06:15:58 [DEBUG   ] Got command DRX from MCS: ref #534309
2015-11-10 06:15:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:15:58 [DEBUG   ] run_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:15:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534309   8 57336 22558019 A  ERROR'
2015-11-10 06:15:58 [DEBUG   ] Got command DRX from MCS: ref #534311
2015-11-10 06:15:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:15:58 [DEBUG   ] run_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:15:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534311   8 57336 22558023 A  ERROR'
2015-11-10 06:15:58 [DEBUG   ] Got command DRX from MCS: ref #534312
2015-11-10 06:15:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:15:58 [DEBUG   ] run_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:15:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534312   8 57336 22558027 A  ERROR'
2015-11-10 06:15:58 [INFO    ] Finished starting DRX
2015-11-10 06:15:58 [INFO    ] Finished starting DRX
2015-11-10 06:15:58 [INFO    ] Finished starting DRX
2015-11-10 06:15:58 [INFO    ] Finished starting DRX
2015-11-10 06:15:58 [DEBUG   ] Got command BAM from MCS: ref #534313
2015-11-10 06:15:58 [INFO    ] Finished processing BAM request
2015-11-10 06:15:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534313   8 57336 22558167 A  ERROR'
2015-11-10 06:15:58 [DEBUG   ] Got command BAM from MCS: ref #534314
2015-11-10 06:15:58 [INFO    ] Finished processing BAM request
2015-11-10 06:15:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534314   8 57336 22558243 A  ERROR'
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 06:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 06:15:58 [DEBUG   ] spawn_process: at 2015-11-10 06:15:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 06:15:59 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:16:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 06:16:02 [DEBUG   ] Finished updating board flags in 1.024 seconds
2015-11-10 06:16:19 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:16:40 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:17:00 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:17:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:17:02 [DEBUG   ] Finished updating board flags in 0.972 seconds
2015-11-10 06:17:20 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2015-11-10 06:17:40 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:17:49 [DEBUG   ] Got command RPT from MCS: ref #534318
2015-11-10 06:17:49 [DEBUG   ] summary = ERROR
2015-11-10 06:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534318  13 57336 22669035 A  ERRORERROR'
2015-11-10 06:17:49 [DEBUG   ] Got command RPT from MCS: ref #534319
2015-11-10 06:17:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534319  77 57336 22669035 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:17:58 [DEBUG   ] Got command DRX from MCS: ref #534335
2015-11-10 06:17:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:17:58 [DEBUG   ] run_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:17:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534335   8 57336 22678015 A  ERROR'
2015-11-10 06:17:58 [DEBUG   ] Got command DRX from MCS: ref #534336
2015-11-10 06:17:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:17:58 [DEBUG   ] run_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:17:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534336   8 57336 22678019 A  ERROR'
2015-11-10 06:17:58 [DEBUG   ] Got command DRX from MCS: ref #534337
2015-11-10 06:17:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:17:58 [DEBUG   ] run_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:17:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534337   8 57336 22678023 A  ERROR'
2015-11-10 06:17:58 [DEBUG   ] Got command DRX from MCS: ref #534338
2015-11-10 06:17:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:17:58 [DEBUG   ] run_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:17:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534338   8 57336 22678027 A  ERROR'
2015-11-10 06:17:58 [INFO    ] Finished starting DRX
2015-11-10 06:17:58 [INFO    ] Finished starting DRX
2015-11-10 06:17:58 [INFO    ] Finished starting DRX
2015-11-10 06:17:58 [INFO    ] Finished starting DRX
2015-11-10 06:17:58 [DEBUG   ] Got command BAM from MCS: ref #534340
2015-11-10 06:17:58 [INFO    ] Finished processing BAM request
2015-11-10 06:17:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534340   8 57336 22678163 A  ERROR'
2015-11-10 06:17:58 [DEBUG   ] Got command BAM from MCS: ref #534339
2015-11-10 06:17:58 [INFO    ] Finished processing BAM request
2015-11-10 06:17:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534339   8 57336 22678263 A  ERROR'
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 06:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 06:17:58 [DEBUG   ] spawn_process: at 2015-11-10 06:17:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 06:18:00 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:18:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:18:03 [DEBUG   ] Finished updating board flags in 1.358 seconds
2015-11-10 06:18:20 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:18:40 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:19:00 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:19:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 06:19:03 [DEBUG   ] Finished updating board flags in 1.204 seconds
2015-11-10 06:19:20 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2015-11-10 06:19:40 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:19:58 [DEBUG   ] Got command DRX from MCS: ref #534341
2015-11-10 06:19:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:19:58 [DEBUG   ] run_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:19:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534341   8 57336 22798011 A  ERROR'
2015-11-10 06:19:58 [DEBUG   ] Got command DRX from MCS: ref #534342
2015-11-10 06:19:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:19:58 [DEBUG   ] run_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:19:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534342   8 57336 22798015 A  ERROR'
2015-11-10 06:19:58 [DEBUG   ] Got command DRX from MCS: ref #534343
2015-11-10 06:19:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:19:58 [DEBUG   ] run_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:19:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534343   8 57336 22798019 A  ERROR'
2015-11-10 06:19:58 [DEBUG   ] Got command DRX from MCS: ref #534344
2015-11-10 06:19:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:19:58 [DEBUG   ] run_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:19:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534344   8 57336 22798023 A  ERROR'
2015-11-10 06:19:58 [INFO    ] Finished starting DRX
2015-11-10 06:19:58 [INFO    ] Finished starting DRX
2015-11-10 06:19:58 [INFO    ] Finished starting DRX
2015-11-10 06:19:58 [INFO    ] Finished starting DRX
2015-11-10 06:19:58 [DEBUG   ] Got command BAM from MCS: ref #534345
2015-11-10 06:19:58 [INFO    ] Finished processing BAM request
2015-11-10 06:19:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534345   8 57336 22798151 A  ERROR'
2015-11-10 06:19:58 [DEBUG   ] Got command BAM from MCS: ref #534346
2015-11-10 06:19:58 [INFO    ] Finished processing BAM request
2015-11-10 06:19:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534346   8 57336 22798251 A  ERROR'
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 06:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 06:19:58 [DEBUG   ] spawn_process: at 2015-11-10 06:19:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 06:20:00 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:20:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:20:03 [DEBUG   ] Finished updating board flags in 1.198 seconds
2015-11-10 06:20:20 [DEBUG   ] Finished updating ant stats in 0.397 seconds
2015-11-10 06:20:25 [DEBUG   ] Got command RPT from MCS: ref #534347
2015-11-10 06:20:25 [DEBUG   ] summary = ERROR
2015-11-10 06:20:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534347  13 57336 22825923 A  ERRORERROR'
2015-11-10 06:20:26 [DEBUG   ] Got command RPT from MCS: ref #534348
2015-11-10 06:20:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:20:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534348  77 57336 22826127 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:20:26 [DEBUG   ] Got command RPT from MCS: ref #534349
2015-11-10 06:20:26 [DEBUG   ] summary = ERROR
2015-11-10 06:20:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534349  13 57336 22826319 A  ERRORERROR'
2015-11-10 06:20:26 [DEBUG   ] Got command RPT from MCS: ref #534350
2015-11-10 06:20:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:20:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534350  77 57336 22826523 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:20:26 [DEBUG   ] Got command RPT from MCS: ref #534351
2015-11-10 06:20:26 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 06:20:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534351  12 57336 22826727 A  ERRORB•™š'
2015-11-10 06:20:41 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:21:01 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 06:21:03 [DEBUG   ] Finished updating board flags in 1.000 seconds
2015-11-10 06:21:21 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:21:41 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:21:58 [DEBUG   ] Got command DRX from MCS: ref #534356
2015-11-10 06:21:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:21:58 [DEBUG   ] run_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:21:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534356   8 57336 22918015 A  ERROR'
2015-11-10 06:21:58 [DEBUG   ] Got command DRX from MCS: ref #534355
2015-11-10 06:21:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:21:58 [DEBUG   ] run_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:21:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534355   8 57336 22918019 A  ERROR'
2015-11-10 06:21:58 [DEBUG   ] Got command DRX from MCS: ref #534358
2015-11-10 06:21:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:21:58 [DEBUG   ] run_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:21:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534358   8 57336 22918023 A  ERROR'
2015-11-10 06:21:58 [DEBUG   ] Got command DRX from MCS: ref #534357
2015-11-10 06:21:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:21:58 [DEBUG   ] run_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:21:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534357   8 57336 22918027 A  ERROR'
2015-11-10 06:21:58 [INFO    ] Finished starting DRX
2015-11-10 06:21:58 [INFO    ] Finished starting DRX
2015-11-10 06:21:58 [INFO    ] Finished starting DRX
2015-11-10 06:21:58 [INFO    ] Finished starting DRX
2015-11-10 06:21:58 [DEBUG   ] Got command BAM from MCS: ref #534360
2015-11-10 06:21:58 [INFO    ] Finished processing BAM request
2015-11-10 06:21:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534360   8 57336 22918179 A  ERROR'
2015-11-10 06:21:58 [DEBUG   ] Got command BAM from MCS: ref #534359
2015-11-10 06:21:58 [INFO    ] Finished processing BAM request
2015-11-10 06:21:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534359   8 57336 22918279 A  ERROR'
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 06:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 06:21:58 [DEBUG   ] spawn_process: at 2015-11-10 06:21:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 06:22:01 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 06:22:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 06:22:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 06:22:03 [DEBUG   ] Finished updating board flags in 0.993 seconds
2015-11-10 06:22:21 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:22:41 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:22:49 [DEBUG   ] Got command RPT from MCS: ref #534361
2015-11-10 06:22:49 [DEBUG   ] summary = ERROR
2015-11-10 06:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534361  13 57336 22969035 A  ERRORERROR'
2015-11-10 06:22:49 [DEBUG   ] Got command RPT from MCS: ref #534362
2015-11-10 06:22:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534362  77 57336 22969035 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:23:01 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 06:23:03 [DEBUG   ] Finished updating board flags in 1.277 seconds
2015-11-10 06:23:21 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:23:41 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 06:23:58 [DEBUG   ] Got command DRX from MCS: ref #534381
2015-11-10 06:23:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:23:58 [DEBUG   ] run_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:23:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534381   8 57336 23038007 A  ERROR'
2015-11-10 06:23:58 [DEBUG   ] Got command DRX from MCS: ref #534382
2015-11-10 06:23:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:23:58 [DEBUG   ] run_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:23:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534382   8 57336 23038011 A  ERROR'
2015-11-10 06:23:58 [INFO    ] Finished starting DRX
2015-11-10 06:23:58 [INFO    ] Finished starting DRX
2015-11-10 06:23:58 [DEBUG   ] Got command DRX from MCS: ref #534383
2015-11-10 06:23:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:23:58 [DEBUG   ] run_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:23:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534383   8 57336 23038147 A  ERROR'
2015-11-10 06:23:58 [INFO    ] Finished starting DRX
2015-11-10 06:23:58 [DEBUG   ] Got command DRX from MCS: ref #534384
2015-11-10 06:23:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:23:58 [DEBUG   ] run_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:23:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534384   8 57336 23038235 A  ERROR'
2015-11-10 06:23:58 [INFO    ] Finished starting DRX
2015-11-10 06:23:58 [DEBUG   ] Got command BAM from MCS: ref #534385
2015-11-10 06:23:58 [INFO    ] Finished processing BAM request
2015-11-10 06:23:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534385   8 57336 23038335 A  ERROR'
2015-11-10 06:23:58 [DEBUG   ] Got command BAM from MCS: ref #534386
2015-11-10 06:23:58 [INFO    ] Finished processing BAM request
2015-11-10 06:23:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534386   8 57336 23038435 A  ERROR'
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 06:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 06:23:58 [DEBUG   ] spawn_process: at 2015-11-10 06:23:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 06:24:01 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 06:24:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:24:04 [DEBUG   ] Finished updating board flags in 1.805 seconds
2015-11-10 06:24:21 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:24:41 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:25:01 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 06:25:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 06:25:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:25:03 [DEBUG   ] Finished updating board flags in 1.191 seconds
2015-11-10 06:25:21 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:25:26 [DEBUG   ] Got command RPT from MCS: ref #534388
2015-11-10 06:25:26 [DEBUG   ] summary = ERROR
2015-11-10 06:25:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534388  13 57336 23126039 A  ERRORERROR'
2015-11-10 06:25:26 [DEBUG   ] Got command RPT from MCS: ref #534389
2015-11-10 06:25:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:25:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534389  77 57336 23126243 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:25:26 [DEBUG   ] Got command RPT from MCS: ref #534390
2015-11-10 06:25:26 [DEBUG   ] summary = ERROR
2015-11-10 06:25:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534390  13 57336 23126435 A  ERRORERROR'
2015-11-10 06:25:26 [DEBUG   ] Got command RPT from MCS: ref #534391
2015-11-10 06:25:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:25:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534391  77 57336 23126639 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:25:26 [DEBUG   ] Got command RPT from MCS: ref #534392
2015-11-10 06:25:26 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 06:25:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534392  12 57336 23126843 A  ERRORB•™š'
2015-11-10 06:25:41 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:25:58 [DEBUG   ] Got command DRX from MCS: ref #534398
2015-11-10 06:25:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:25:58 [DEBUG   ] run_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:25:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534398   8 57336 23158011 A  ERROR'
2015-11-10 06:25:58 [DEBUG   ] Got command DRX from MCS: ref #534399
2015-11-10 06:25:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:25:58 [DEBUG   ] run_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:25:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534399   8 57336 23158015 A  ERROR'
2015-11-10 06:25:58 [DEBUG   ] Got command DRX from MCS: ref #534400
2015-11-10 06:25:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:25:58 [DEBUG   ] run_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:25:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534400   8 57336 23158019 A  ERROR'
2015-11-10 06:25:58 [DEBUG   ] Got command DRX from MCS: ref #534401
2015-11-10 06:25:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:25:58 [DEBUG   ] run_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:25:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534401   8 57336 23158023 A  ERROR'
2015-11-10 06:25:58 [INFO    ] Finished starting DRX
2015-11-10 06:25:58 [INFO    ] Finished starting DRX
2015-11-10 06:25:58 [INFO    ] Finished starting DRX
2015-11-10 06:25:58 [INFO    ] Finished starting DRX
2015-11-10 06:25:58 [DEBUG   ] Got command BAM from MCS: ref #534403
2015-11-10 06:25:58 [INFO    ] Finished processing BAM request
2015-11-10 06:25:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534403   8 57336 23158187 A  ERROR'
2015-11-10 06:25:58 [DEBUG   ] Got command BAM from MCS: ref #534402
2015-11-10 06:25:58 [INFO    ] Finished processing BAM request
2015-11-10 06:25:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534402   8 57336 23158287 A  ERROR'
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 06:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 06:25:58 [DEBUG   ] spawn_process: at 2015-11-10 06:25:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 06:26:01 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 06:26:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 06:26:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:26:03 [DEBUG   ] Finished updating board flags in 1.004 seconds
2015-11-10 06:26:21 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 06:26:41 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:27:01 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2015-11-10 06:27:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 06:27:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 06:27:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 06:27:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:27:03 [DEBUG   ] Finished updating board flags in 1.045 seconds
2015-11-10 06:27:21 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:27:42 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:27:49 [DEBUG   ] Got command RPT from MCS: ref #534407
2015-11-10 06:27:49 [DEBUG   ] summary = ERROR
2015-11-10 06:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534407  13 57336 23269039 A  ERRORERROR'
2015-11-10 06:27:49 [DEBUG   ] Got command RPT from MCS: ref #534408
2015-11-10 06:27:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534408  77 57336 23269039 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:27:58 [DEBUG   ] Got command DRX from MCS: ref #534424
2015-11-10 06:27:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:27:58 [DEBUG   ] run_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:27:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534424   8 57336 23278007 A  ERROR'
2015-11-10 06:27:58 [DEBUG   ] Got command DRX from MCS: ref #534425
2015-11-10 06:27:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:27:58 [DEBUG   ] run_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:27:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534425   8 57336 23278011 A  ERROR'
2015-11-10 06:27:58 [INFO    ] Finished starting DRX
2015-11-10 06:27:58 [INFO    ] Finished starting DRX
2015-11-10 06:27:58 [DEBUG   ] Got command DRX from MCS: ref #534426
2015-11-10 06:27:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:27:58 [DEBUG   ] run_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:27:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534426   8 57336 23278183 A  ERROR'
2015-11-10 06:27:58 [INFO    ] Finished starting DRX
2015-11-10 06:27:58 [DEBUG   ] Got command DRX from MCS: ref #534427
2015-11-10 06:27:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:27:58 [DEBUG   ] run_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:27:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534427   8 57336 23278271 A  ERROR'
2015-11-10 06:27:58 [INFO    ] Finished starting DRX
2015-11-10 06:27:58 [DEBUG   ] Got command BAM from MCS: ref #534429
2015-11-10 06:27:58 [INFO    ] Finished processing BAM request
2015-11-10 06:27:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534429   8 57336 23278431 A  ERROR'
2015-11-10 06:27:58 [DEBUG   ] Got command BAM from MCS: ref #534428
2015-11-10 06:27:58 [INFO    ] Finished processing BAM request
2015-11-10 06:27:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534428   8 57336 23278507 A  ERROR'
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 06:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 06:27:58 [DEBUG   ] spawn_process: at 2015-11-10 06:27:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 06:28:02 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:28:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 06:28:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 06:28:03 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 06:28:03 [DEBUG   ] Finished updating board flags in 0.949 seconds
2015-11-10 06:28:22 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 06:28:42 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:29:02 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:29:04 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 06:29:04 [DEBUG   ] Finished updating board flags in 1.349 seconds
2015-11-10 06:29:22 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 06:29:42 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 06:29:58 [DEBUG   ] Got command DRX from MCS: ref #534433
2015-11-10 06:29:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:29:58 [DEBUG   ] run_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:29:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534433   8 57336 23398015 A  ERROR'
2015-11-10 06:29:58 [DEBUG   ] Got command DRX from MCS: ref #534434
2015-11-10 06:29:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:29:58 [DEBUG   ] run_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:29:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534434   8 57336 23398019 A  ERROR'
2015-11-10 06:29:58 [DEBUG   ] Got command DRX from MCS: ref #534435
2015-11-10 06:29:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:29:58 [DEBUG   ] run_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:29:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534435   8 57336 23398023 A  ERROR'
2015-11-10 06:29:58 [DEBUG   ] Got command DRX from MCS: ref #534436
2015-11-10 06:29:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:29:58 [DEBUG   ] run_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:29:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534436   8 57336 23398027 A  ERROR'
2015-11-10 06:29:58 [INFO    ] Finished starting DRX
2015-11-10 06:29:58 [INFO    ] Finished starting DRX
2015-11-10 06:29:58 [INFO    ] Finished starting DRX
2015-11-10 06:29:58 [INFO    ] Finished starting DRX
2015-11-10 06:29:58 [DEBUG   ] Got command BAM from MCS: ref #534438
2015-11-10 06:29:58 [INFO    ] Finished processing BAM request
2015-11-10 06:29:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534438   8 57336 23398163 A  ERROR'
2015-11-10 06:29:58 [DEBUG   ] Got command BAM from MCS: ref #534437
2015-11-10 06:29:58 [INFO    ] Finished processing BAM request
2015-11-10 06:29:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534437   8 57336 23398287 A  ERROR'
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 06:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 06:29:58 [DEBUG   ] spawn_process: at 2015-11-10 06:29:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 06:30:02 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 06:30:04 [DEBUG   ] Finished updating board flags in 1.268 seconds
2015-11-10 06:30:22 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 06:30:26 [DEBUG   ] Got command RPT from MCS: ref #534439
2015-11-10 06:30:26 [DEBUG   ] summary = ERROR
2015-11-10 06:30:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534439  13 57336 23426147 A  ERRORERROR'
2015-11-10 06:30:26 [DEBUG   ] Got command RPT from MCS: ref #534440
2015-11-10 06:30:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:30:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534440  77 57336 23426351 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:30:26 [DEBUG   ] Got command RPT from MCS: ref #534441
2015-11-10 06:30:26 [DEBUG   ] summary = ERROR
2015-11-10 06:30:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534441  13 57336 23426555 A  ERRORERROR'
2015-11-10 06:30:26 [DEBUG   ] Got command RPT from MCS: ref #534442
2015-11-10 06:30:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:30:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534442  77 57336 23426747 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:30:26 [DEBUG   ] Got command RPT from MCS: ref #534443
2015-11-10 06:30:26 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 06:30:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534443  12 57336 23426951 A  ERRORB•™š'
2015-11-10 06:30:42 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:31:02 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:31:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 06:31:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:31:04 [DEBUG   ] Finished updating board flags in 1.162 seconds
2015-11-10 06:31:22 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 06:31:42 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:31:58 [DEBUG   ] Got command DRX from MCS: ref #534444
2015-11-10 06:31:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:31:58 [DEBUG   ] run_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:31:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534444   8 57336 23518015 A  ERROR'
2015-11-10 06:31:58 [DEBUG   ] Got command DRX from MCS: ref #534445
2015-11-10 06:31:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:31:58 [DEBUG   ] run_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:31:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534445   8 57336 23518019 A  ERROR'
2015-11-10 06:31:58 [DEBUG   ] Got command DRX from MCS: ref #534446
2015-11-10 06:31:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:31:58 [DEBUG   ] run_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:31:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534446   8 57336 23518023 A  ERROR'
2015-11-10 06:31:58 [DEBUG   ] Got command DRX from MCS: ref #534447
2015-11-10 06:31:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:31:58 [DEBUG   ] run_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:31:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534447   8 57336 23518027 A  ERROR'
2015-11-10 06:31:58 [INFO    ] Finished starting DRX
2015-11-10 06:31:58 [INFO    ] Finished starting DRX
2015-11-10 06:31:58 [INFO    ] Finished starting DRX
2015-11-10 06:31:58 [INFO    ] Finished starting DRX
2015-11-10 06:31:58 [DEBUG   ] Got command BAM from MCS: ref #534449
2015-11-10 06:31:58 [INFO    ] Finished processing BAM request
2015-11-10 06:31:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534449   8 57336 23518203 A  ERROR'
2015-11-10 06:31:58 [DEBUG   ] Got command BAM from MCS: ref #534448
2015-11-10 06:31:58 [INFO    ] Finished processing BAM request
2015-11-10 06:31:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534448   8 57336 23518303 A  ERROR'
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 06:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 06:31:58 [DEBUG   ] spawn_process: at 2015-11-10 06:31:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 06:32:02 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:32:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2015-11-10 06:32:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:32:05 [DEBUG   ] Finished updating board flags in 2.001 seconds
2015-11-10 06:32:22 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-10 06:32:42 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:32:49 [DEBUG   ] Got command RPT from MCS: ref #534453
2015-11-10 06:32:49 [DEBUG   ] summary = ERROR
2015-11-10 06:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534453  13 57336 23569035 A  ERRORERROR'
2015-11-10 06:32:49 [DEBUG   ] Got command RPT from MCS: ref #534454
2015-11-10 06:32:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534454  77 57336 23569035 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:33:02 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 06:33:05 [DEBUG   ] Finished updating board flags in 1.961 seconds
2015-11-10 06:33:22 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 06:33:42 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:33:58 [DEBUG   ] Got command DRX from MCS: ref #534470
2015-11-10 06:33:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:33:58 [DEBUG   ] run_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:33:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534470   8 57336 23638011 A  ERROR'
2015-11-10 06:33:58 [DEBUG   ] Got command DRX from MCS: ref #534471
2015-11-10 06:33:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:33:58 [DEBUG   ] run_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:33:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534471   8 57336 23638015 A  ERROR'
2015-11-10 06:33:58 [DEBUG   ] Got command DRX from MCS: ref #534472
2015-11-10 06:33:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:33:58 [DEBUG   ] run_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:33:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534472   8 57336 23638019 A  ERROR'
2015-11-10 06:33:58 [DEBUG   ] Got command DRX from MCS: ref #534473
2015-11-10 06:33:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:33:58 [DEBUG   ] run_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:33:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534473   8 57336 23638023 A  ERROR'
2015-11-10 06:33:58 [INFO    ] Finished starting DRX
2015-11-10 06:33:58 [INFO    ] Finished starting DRX
2015-11-10 06:33:58 [INFO    ] Finished starting DRX
2015-11-10 06:33:58 [INFO    ] Finished starting DRX
2015-11-10 06:33:58 [DEBUG   ] Got command BAM from MCS: ref #534475
2015-11-10 06:33:58 [INFO    ] Finished processing BAM request
2015-11-10 06:33:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534475   8 57336 23638219 A  ERROR'
2015-11-10 06:33:58 [DEBUG   ] Got command BAM from MCS: ref #534474
2015-11-10 06:33:58 [INFO    ] Finished processing BAM request
2015-11-10 06:33:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534474   8 57336 23638319 A  ERROR'
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 06:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 06:33:58 [DEBUG   ] spawn_process: at 2015-11-10 06:33:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 06:34:02 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:34:04 [DEBUG   ] Finished updating board flags in 1.250 seconds
2015-11-10 06:34:22 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 06:34:42 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 06:35:02 [DEBUG   ] Finished updating ant stats in 1.223 seconds
2015-11-10 06:35:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 06:35:05 [DEBUG   ] Finished updating board flags in 2.246 seconds
2015-11-10 06:35:22 [DEBUG   ] Finished updating ant stats in 0.981 seconds
2015-11-10 06:35:26 [DEBUG   ] Got command RPT from MCS: ref #534477
2015-11-10 06:35:26 [DEBUG   ] summary = ERROR
2015-11-10 06:35:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534477  13 57336 23726247 A  ERRORERROR'
2015-11-10 06:35:26 [DEBUG   ] Got command RPT from MCS: ref #534478
2015-11-10 06:35:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:35:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534478  77 57336 23726451 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:35:26 [DEBUG   ] Got command RPT from MCS: ref #534479
2015-11-10 06:35:26 [DEBUG   ] summary = ERROR
2015-11-10 06:35:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534479  13 57336 23726655 A  ERRORERROR'
2015-11-10 06:35:26 [DEBUG   ] Got command RPT from MCS: ref #534480
2015-11-10 06:35:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:35:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534480  77 57336 23726859 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:35:27 [DEBUG   ] Got command RPT from MCS: ref #534481
2015-11-10 06:35:27 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 06:35:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534481  12 57336 23727051 A  ERRORB•™š'
2015-11-10 06:35:42 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:35:58 [DEBUG   ] Got command DRX from MCS: ref #534491
2015-11-10 06:35:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:35:58 [DEBUG   ] run_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:35:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534491   8 57336 23758007 A  ERROR'
2015-11-10 06:35:58 [DEBUG   ] Got command DRX from MCS: ref #534490
2015-11-10 06:35:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:35:58 [DEBUG   ] run_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:35:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534490   8 57336 23758011 A  ERROR'
2015-11-10 06:35:58 [INFO    ] Finished starting DRX
2015-11-10 06:35:58 [INFO    ] Finished starting DRX
2015-11-10 06:35:58 [DEBUG   ] Got command DRX from MCS: ref #534492
2015-11-10 06:35:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:35:58 [DEBUG   ] run_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:35:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534492   8 57336 23758171 A  ERROR'
2015-11-10 06:35:58 [INFO    ] Finished starting DRX
2015-11-10 06:35:58 [DEBUG   ] Got command DRX from MCS: ref #534493
2015-11-10 06:35:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:35:58 [DEBUG   ] run_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:35:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534493   8 57336 23758283 A  ERROR'
2015-11-10 06:35:58 [INFO    ] Finished starting DRX
2015-11-10 06:35:58 [DEBUG   ] Got command BAM from MCS: ref #534494
2015-11-10 06:35:58 [INFO    ] Finished processing BAM request
2015-11-10 06:35:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534494   8 57336 23758431 A  ERROR'
2015-11-10 06:35:58 [DEBUG   ] Got command BAM from MCS: ref #534495
2015-11-10 06:35:58 [INFO    ] Finished processing BAM request
2015-11-10 06:35:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534495   8 57336 23758543 A  ERROR'
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 06:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 06:35:58 [DEBUG   ] spawn_process: at 2015-11-10 06:35:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 06:36:02 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:36:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 06:36:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 06:36:04 [DEBUG   ] Finished updating board flags in 1.152 seconds
2015-11-10 06:36:22 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 06:36:42 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:37:02 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:37:04 [DEBUG   ] Finished updating board flags in 1.074 seconds
2015-11-10 06:37:22 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 06:37:42 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:37:49 [DEBUG   ] Got command RPT from MCS: ref #534496
2015-11-10 06:37:49 [DEBUG   ] summary = ERROR
2015-11-10 06:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534496  13 57336 23869035 A  ERRORERROR'
2015-11-10 06:37:49 [DEBUG   ] Got command RPT from MCS: ref #534497
2015-11-10 06:37:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534497  77 57336 23869035 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:37:58 [DEBUG   ] Got command DRX from MCS: ref #534513
2015-11-10 06:37:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:37:58 [DEBUG   ] run_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:37:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534513   8 57336 23878015 A  ERROR'
2015-11-10 06:37:58 [DEBUG   ] Got command DRX from MCS: ref #534514
2015-11-10 06:37:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:37:58 [DEBUG   ] run_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:37:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534514   8 57336 23878019 A  ERROR'
2015-11-10 06:37:58 [DEBUG   ] Got command DRX from MCS: ref #534515
2015-11-10 06:37:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:37:58 [DEBUG   ] run_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:37:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534515   8 57336 23878023 A  ERROR'
2015-11-10 06:37:58 [DEBUG   ] Got command DRX from MCS: ref #534516
2015-11-10 06:37:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:37:58 [DEBUG   ] run_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:37:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534516   8 57336 23878027 A  ERROR'
2015-11-10 06:37:58 [INFO    ] Finished starting DRX
2015-11-10 06:37:58 [INFO    ] Finished starting DRX
2015-11-10 06:37:58 [INFO    ] Finished starting DRX
2015-11-10 06:37:58 [INFO    ] Finished starting DRX
2015-11-10 06:37:58 [DEBUG   ] Got command BAM from MCS: ref #534517
2015-11-10 06:37:58 [INFO    ] Finished processing BAM request
2015-11-10 06:37:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534517   8 57336 23878179 A  ERROR'
2015-11-10 06:37:58 [DEBUG   ] Got command BAM from MCS: ref #534518
2015-11-10 06:37:58 [INFO    ] Finished processing BAM request
2015-11-10 06:37:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534518   8 57336 23878279 A  ERROR'
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 06:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 06:37:58 [DEBUG   ] spawn_process: at 2015-11-10 06:37:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 06:38:02 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:38:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2015-11-10 06:38:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 06:38:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:38:04 [DEBUG   ] Finished updating board flags in 0.969 seconds
2015-11-10 06:38:22 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 06:38:42 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 06:39:02 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:39:04 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 06:39:05 [DEBUG   ] Finished updating board flags in 1.370 seconds
2015-11-10 06:39:22 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 06:39:43 [DEBUG   ] Finished updating ant stats in 1.800 seconds
2015-11-10 06:39:58 [DEBUG   ] Got command DRX from MCS: ref #534523
2015-11-10 06:39:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:39:58 [DEBUG   ] run_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534523   8 57336 23998015 A  ERROR'
2015-11-10 06:39:58 [DEBUG   ] Got command DRX from MCS: ref #534522
2015-11-10 06:39:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:39:58 [DEBUG   ] run_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534522   8 57336 23998019 A  ERROR'
2015-11-10 06:39:58 [DEBUG   ] Got command DRX from MCS: ref #534524
2015-11-10 06:39:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:39:58 [DEBUG   ] run_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534524   8 57336 23998023 A  ERROR'
2015-11-10 06:39:58 [DEBUG   ] Got command DRX from MCS: ref #534525
2015-11-10 06:39:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:39:58 [DEBUG   ] run_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534525   8 57336 23998027 A  ERROR'
2015-11-10 06:39:58 [INFO    ] Finished starting DRX
2015-11-10 06:39:58 [INFO    ] Finished starting DRX
2015-11-10 06:39:58 [INFO    ] Finished starting DRX
2015-11-10 06:39:58 [INFO    ] Finished starting DRX
2015-11-10 06:39:58 [DEBUG   ] Got command BAM from MCS: ref #534527
2015-11-10 06:39:58 [INFO    ] Finished processing BAM request
2015-11-10 06:39:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534527   8 57336 23998191 A  ERROR'
2015-11-10 06:39:58 [DEBUG   ] Got command BAM from MCS: ref #534526
2015-11-10 06:39:58 [INFO    ] Finished processing BAM request
2015-11-10 06:39:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534526   8 57336 23998315 A  ERROR'
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 06:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 06:39:58 [DEBUG   ] spawn_process: at 2015-11-10 06:39:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 06:40:02 [DEBUG   ] Finished updating ant stats in 1.037 seconds
2015-11-10 06:40:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 06:40:05 [DEBUG   ] Finished updating board flags in 1.208 seconds
2015-11-10 06:40:23 [DEBUG   ] Finished updating ant stats in 1.764 seconds
2015-11-10 06:40:26 [DEBUG   ] Got command RPT from MCS: ref #534528
2015-11-10 06:40:26 [DEBUG   ] summary = ERROR
2015-11-10 06:40:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534528  13 57336 24026391 A  ERRORERROR'
2015-11-10 06:40:26 [DEBUG   ] Got command RPT from MCS: ref #534529
2015-11-10 06:40:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:40:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534529  77 57336 24026583 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:40:26 [DEBUG   ] Got command RPT from MCS: ref #534530
2015-11-10 06:40:26 [DEBUG   ] summary = ERROR
2015-11-10 06:40:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534530  13 57336 24026787 A  ERRORERROR'
2015-11-10 06:40:26 [DEBUG   ] Got command RPT from MCS: ref #534531
2015-11-10 06:40:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:40:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534531  77 57336 24026991 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:40:27 [DEBUG   ] Got command RPT from MCS: ref #534532
2015-11-10 06:40:27 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 06:40:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534532  12 57336 24027195 A  ERRORB•™š'
2015-11-10 06:40:42 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:41:02 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:41:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 06:41:05 [DEBUG   ] Finished updating board flags in 1.138 seconds
2015-11-10 06:41:22 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:41:43 [DEBUG   ] Finished updating ant stats in 1.593 seconds
2015-11-10 06:41:58 [DEBUG   ] Got command DRX from MCS: ref #534537
2015-11-10 06:41:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:41:58 [DEBUG   ] run_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:41:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534537   8 57336 24118011 A  ERROR'
2015-11-10 06:41:58 [DEBUG   ] Got command DRX from MCS: ref #534536
2015-11-10 06:41:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:41:58 [DEBUG   ] run_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:41:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534536   8 57336 24118015 A  ERROR'
2015-11-10 06:41:58 [DEBUG   ] Got command DRX from MCS: ref #534538
2015-11-10 06:41:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:41:58 [DEBUG   ] run_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:41:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534538   8 57336 24118019 A  ERROR'
2015-11-10 06:41:58 [DEBUG   ] Got command DRX from MCS: ref #534539
2015-11-10 06:41:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:41:58 [DEBUG   ] run_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:41:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534539   8 57336 24118023 A  ERROR'
2015-11-10 06:41:58 [INFO    ] Finished starting DRX
2015-11-10 06:41:58 [INFO    ] Finished starting DRX
2015-11-10 06:41:58 [INFO    ] Finished starting DRX
2015-11-10 06:41:58 [INFO    ] Finished starting DRX
2015-11-10 06:41:58 [DEBUG   ] Got command BAM from MCS: ref #534541
2015-11-10 06:41:58 [INFO    ] Finished processing BAM request
2015-11-10 06:41:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534541   8 57336 24118159 A  ERROR'
2015-11-10 06:41:58 [DEBUG   ] Got command BAM from MCS: ref #534540
2015-11-10 06:41:58 [INFO    ] Finished processing BAM request
2015-11-10 06:41:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534540   8 57336 24118295 A  ERROR'
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 06:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 06:41:58 [DEBUG   ] spawn_process: at 2015-11-10 06:41:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 06:42:02 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2015-11-10 06:42:05 [DEBUG   ] Finished updating board flags in 1.007 seconds
2015-11-10 06:42:22 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 06:42:42 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 06:42:49 [DEBUG   ] Got command RPT from MCS: ref #534542
2015-11-10 06:42:49 [DEBUG   ] summary = ERROR
2015-11-10 06:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534542  13 57336 24169043 A  ERRORERROR'
2015-11-10 06:42:49 [DEBUG   ] Got command RPT from MCS: ref #534543
2015-11-10 06:42:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534543  77 57336 24169043 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:43:02 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 06:43:05 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:43:05 [DEBUG   ] Finished updating board flags in 1.018 seconds
2015-11-10 06:43:22 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 06:43:42 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:43:58 [DEBUG   ] Got command DRX from MCS: ref #534559
2015-11-10 06:43:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:43:58 [DEBUG   ] run_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:43:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534559   8 57336 24238011 A  ERROR'
2015-11-10 06:43:58 [DEBUG   ] Got command DRX from MCS: ref #534560
2015-11-10 06:43:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:43:58 [DEBUG   ] run_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:43:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534560   8 57336 24238015 A  ERROR'
2015-11-10 06:43:58 [DEBUG   ] Got command DRX from MCS: ref #534561
2015-11-10 06:43:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:43:58 [DEBUG   ] run_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:43:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534561   8 57336 24238019 A  ERROR'
2015-11-10 06:43:58 [DEBUG   ] Got command DRX from MCS: ref #534562
2015-11-10 06:43:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:43:58 [DEBUG   ] run_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:43:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534562   8 57336 24238023 A  ERROR'
2015-11-10 06:43:58 [INFO    ] Finished starting DRX
2015-11-10 06:43:58 [INFO    ] Finished starting DRX
2015-11-10 06:43:58 [INFO    ] Finished starting DRX
2015-11-10 06:43:58 [INFO    ] Finished starting DRX
2015-11-10 06:43:58 [DEBUG   ] Got command BAM from MCS: ref #534563
2015-11-10 06:43:58 [INFO    ] Finished processing BAM request
2015-11-10 06:43:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534563   8 57336 24238147 A  ERROR'
2015-11-10 06:43:58 [DEBUG   ] Got command BAM from MCS: ref #534564
2015-11-10 06:43:58 [INFO    ] Finished processing BAM request
2015-11-10 06:43:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534564   8 57336 24238247 A  ERROR'
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 06:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 06:43:58 [DEBUG   ] spawn_process: at 2015-11-10 06:43:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 06:44:02 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 06:44:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa34, chip 1
2015-11-10 06:44:05 [DEBUG   ] Finished updating board flags in 1.020 seconds
2015-11-10 06:44:22 [DEBUG   ] Finished updating ant stats in 0.397 seconds
2015-11-10 06:44:43 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:45:03 [DEBUG   ] Finished updating ant stats in 1.342 seconds
2015-11-10 06:45:06 [DEBUG   ] Finished updating board flags in 1.781 seconds
2015-11-10 06:45:23 [DEBUG   ] Finished updating ant stats in 1.060 seconds
2015-11-10 06:45:26 [DEBUG   ] Got command RPT from MCS: ref #534569
2015-11-10 06:45:26 [DEBUG   ] summary = ERROR
2015-11-10 06:45:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534569  13 57336 24326487 A  ERRORERROR'
2015-11-10 06:45:26 [DEBUG   ] Got command RPT from MCS: ref #534570
2015-11-10 06:45:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:45:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534570  77 57336 24326691 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:45:26 [DEBUG   ] Got command RPT from MCS: ref #534571
2015-11-10 06:45:26 [DEBUG   ] summary = ERROR
2015-11-10 06:45:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534571  13 57336 24326895 A  ERRORERROR'
2015-11-10 06:45:27 [DEBUG   ] Got command RPT from MCS: ref #534572
2015-11-10 06:45:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:45:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534572  77 57336 24327099 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:45:27 [DEBUG   ] Got command RPT from MCS: ref #534573
2015-11-10 06:45:27 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 06:45:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534573  12 57336 24327291 A  ERRORB–™š'
2015-11-10 06:45:43 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:45:58 [DEBUG   ] Got command DRX from MCS: ref #534580
2015-11-10 06:45:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:45:58 [DEBUG   ] run_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:45:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534580   8 57336 24358015 A  ERROR'
2015-11-10 06:45:58 [DEBUG   ] Got command DRX from MCS: ref #534579
2015-11-10 06:45:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:45:58 [DEBUG   ] run_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:45:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534579   8 57336 24358019 A  ERROR'
2015-11-10 06:45:58 [DEBUG   ] Got command DRX from MCS: ref #534581
2015-11-10 06:45:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:45:58 [DEBUG   ] run_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:45:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534581   8 57336 24358023 A  ERROR'
2015-11-10 06:45:58 [DEBUG   ] Got command DRX from MCS: ref #534582
2015-11-10 06:45:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:45:58 [DEBUG   ] run_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:45:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534582   8 57336 24358027 A  ERROR'
2015-11-10 06:45:58 [INFO    ] Finished starting DRX
2015-11-10 06:45:58 [INFO    ] Finished starting DRX
2015-11-10 06:45:58 [INFO    ] Finished starting DRX
2015-11-10 06:45:58 [INFO    ] Finished starting DRX
2015-11-10 06:45:58 [DEBUG   ] Got command BAM from MCS: ref #534584
2015-11-10 06:45:58 [INFO    ] Finished processing BAM request
2015-11-10 06:45:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534584   8 57336 24358191 A  ERROR'
2015-11-10 06:45:58 [DEBUG   ] Got command BAM from MCS: ref #534583
2015-11-10 06:45:58 [INFO    ] Finished processing BAM request
2015-11-10 06:45:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534583   8 57336 24358303 A  ERROR'
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 06:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 06:45:58 [DEBUG   ] spawn_process: at 2015-11-10 06:45:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 06:46:03 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 06:46:05 [DEBUG   ] Finished updating board flags in 1.238 seconds
2015-11-10 06:46:23 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:46:43 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:47:03 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:47:05 [DEBUG   ] Finished updating board flags in 1.229 seconds
2015-11-10 06:47:23 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:47:43 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:47:49 [DEBUG   ] Got command RPT from MCS: ref #534588
2015-11-10 06:47:49 [DEBUG   ] summary = ERROR
2015-11-10 06:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534588  13 57336 24469047 A  ERRORERROR'
2015-11-10 06:47:49 [DEBUG   ] Got command RPT from MCS: ref #534589
2015-11-10 06:47:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534589  77 57336 24469047 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:47:58 [DEBUG   ] Got command DRX from MCS: ref #534606
2015-11-10 06:47:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:47:58 [DEBUG   ] run_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:47:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534606   8 57336 24478015 A  ERROR'
2015-11-10 06:47:58 [DEBUG   ] Got command DRX from MCS: ref #534605
2015-11-10 06:47:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:47:58 [DEBUG   ] run_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:47:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534605   8 57336 24478019 A  ERROR'
2015-11-10 06:47:58 [DEBUG   ] Got command DRX from MCS: ref #534607
2015-11-10 06:47:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:47:58 [DEBUG   ] run_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:47:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534607   8 57336 24478023 A  ERROR'
2015-11-10 06:47:58 [DEBUG   ] Got command DRX from MCS: ref #534608
2015-11-10 06:47:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:47:58 [DEBUG   ] run_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:47:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534608   8 57336 24478027 A  ERROR'
2015-11-10 06:47:58 [INFO    ] Finished starting DRX
2015-11-10 06:47:58 [INFO    ] Finished starting DRX
2015-11-10 06:47:58 [INFO    ] Finished starting DRX
2015-11-10 06:47:58 [INFO    ] Finished starting DRX
2015-11-10 06:47:58 [DEBUG   ] Got command BAM from MCS: ref #534609
2015-11-10 06:47:58 [INFO    ] Finished processing BAM request
2015-11-10 06:47:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534609   8 57336 24478199 A  ERROR'
2015-11-10 06:47:58 [DEBUG   ] Got command BAM from MCS: ref #534610
2015-11-10 06:47:58 [INFO    ] Finished processing BAM request
2015-11-10 06:47:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534610   8 57336 24478311 A  ERROR'
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 06:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 06:47:58 [DEBUG   ] spawn_process: at 2015-11-10 06:47:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 06:48:03 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 06:48:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 06:48:06 [DEBUG   ] Finished updating board flags in 1.759 seconds
2015-11-10 06:48:23 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:48:43 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:49:03 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:49:05 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 06:49:05 [DEBUG   ] Finished updating board flags in 1.034 seconds
2015-11-10 06:49:23 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:49:43 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:49:58 [DEBUG   ] Got command DRX from MCS: ref #534611
2015-11-10 06:49:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:49:58 [DEBUG   ] run_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:49:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534611   8 57336 24598007 A  ERROR'
2015-11-10 06:49:58 [DEBUG   ] Got command DRX from MCS: ref #534612
2015-11-10 06:49:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:49:58 [DEBUG   ] run_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:49:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534612   8 57336 24598011 A  ERROR'
2015-11-10 06:49:58 [INFO    ] Finished starting DRX
2015-11-10 06:49:58 [INFO    ] Finished starting DRX
2015-11-10 06:49:58 [DEBUG   ] Got command DRX from MCS: ref #534613
2015-11-10 06:49:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:49:58 [DEBUG   ] run_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:49:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534613   8 57336 24598159 A  ERROR'
2015-11-10 06:49:58 [INFO    ] Finished starting DRX
2015-11-10 06:49:58 [DEBUG   ] Got command DRX from MCS: ref #534614
2015-11-10 06:49:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:49:58 [DEBUG   ] run_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:49:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534614   8 57336 24598283 A  ERROR'
2015-11-10 06:49:58 [INFO    ] Finished starting DRX
2015-11-10 06:49:58 [DEBUG   ] Got command BAM from MCS: ref #534616
2015-11-10 06:49:58 [INFO    ] Finished processing BAM request
2015-11-10 06:49:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534616   8 57336 24598407 A  ERROR'
2015-11-10 06:49:58 [DEBUG   ] Got command BAM from MCS: ref #534615
2015-11-10 06:49:58 [INFO    ] Finished processing BAM request
2015-11-10 06:49:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534615   8 57336 24598519 A  ERROR'
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 06:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 06:49:58 [DEBUG   ] spawn_process: at 2015-11-10 06:49:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 06:50:03 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 06:50:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:50:06 [DEBUG   ] Finished updating board flags in 1.549 seconds
2015-11-10 06:50:23 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2015-11-10 06:50:26 [DEBUG   ] Got command RPT from MCS: ref #534617
2015-11-10 06:50:26 [DEBUG   ] summary = ERROR
2015-11-10 06:50:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534617  13 57336 24626587 A  ERRORERROR'
2015-11-10 06:50:26 [DEBUG   ] Got command RPT from MCS: ref #534618
2015-11-10 06:50:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:50:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534618  77 57336 24626791 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:50:26 [DEBUG   ] Got command RPT from MCS: ref #534619
2015-11-10 06:50:26 [DEBUG   ] summary = ERROR
2015-11-10 06:50:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534619  13 57336 24626983 A  ERRORERROR'
2015-11-10 06:50:27 [DEBUG   ] Got command RPT from MCS: ref #534620
2015-11-10 06:50:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:50:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534620  77 57336 24627187 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:50:27 [DEBUG   ] Got command RPT from MCS: ref #534621
2015-11-10 06:50:27 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 06:50:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534621  12 57336 24627391 A  ERRORB•™š'
2015-11-10 06:50:43 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 06:51:03 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 06:51:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 06:51:06 [DEBUG   ] Finished updating board flags in 1.412 seconds
2015-11-10 06:51:23 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:51:43 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:51:58 [DEBUG   ] Got command DRX from MCS: ref #534626
2015-11-10 06:51:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:51:58 [DEBUG   ] run_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:51:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534626   8 57336 24718011 A  ERROR'
2015-11-10 06:51:58 [DEBUG   ] Got command DRX from MCS: ref #534625
2015-11-10 06:51:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:51:58 [DEBUG   ] run_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:51:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534625   8 57336 24718015 A  ERROR'
2015-11-10 06:51:58 [DEBUG   ] Got command DRX from MCS: ref #534628
2015-11-10 06:51:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:51:58 [DEBUG   ] run_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:51:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534628   8 57336 24718019 A  ERROR'
2015-11-10 06:51:58 [DEBUG   ] Got command DRX from MCS: ref #534627
2015-11-10 06:51:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:51:58 [DEBUG   ] run_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:51:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534627   8 57336 24718023 A  ERROR'
2015-11-10 06:51:58 [INFO    ] Finished starting DRX
2015-11-10 06:51:58 [INFO    ] Finished starting DRX
2015-11-10 06:51:58 [INFO    ] Finished starting DRX
2015-11-10 06:51:58 [INFO    ] Finished starting DRX
2015-11-10 06:51:58 [DEBUG   ] Got command BAM from MCS: ref #534629
2015-11-10 06:51:58 [INFO    ] Finished processing BAM request
2015-11-10 06:51:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534629   8 57336 24718175 A  ERROR'
2015-11-10 06:51:58 [DEBUG   ] Got command BAM from MCS: ref #534630
2015-11-10 06:51:58 [INFO    ] Finished processing BAM request
2015-11-10 06:51:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534630   8 57336 24718299 A  ERROR'
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 06:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 06:51:58 [DEBUG   ] spawn_process: at 2015-11-10 06:51:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 06:52:03 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 06:52:06 [DEBUG   ] Finished updating board flags in 1.388 seconds
2015-11-10 06:52:23 [DEBUG   ] Finished updating ant stats in 0.396 seconds
2015-11-10 06:52:44 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 06:52:49 [DEBUG   ] Got command RPT from MCS: ref #534631
2015-11-10 06:52:49 [DEBUG   ] summary = ERROR
2015-11-10 06:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534631  13 57336 24769039 A  ERRORERROR'
2015-11-10 06:52:49 [DEBUG   ] Got command RPT from MCS: ref #534632
2015-11-10 06:52:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534632  77 57336 24769039 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:53:04 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 06:53:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:53:06 [DEBUG   ] Finished updating board flags in 1.219 seconds
2015-11-10 06:53:24 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2015-11-10 06:53:44 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 06:53:58 [DEBUG   ] Got command DRX from MCS: ref #534651
2015-11-10 06:53:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:53:58 [DEBUG   ] run_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:53:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534651   8 57336 24838011 A  ERROR'
2015-11-10 06:53:58 [DEBUG   ] Got command DRX from MCS: ref #534652
2015-11-10 06:53:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:53:58 [DEBUG   ] run_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:53:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534652   8 57336 24838015 A  ERROR'
2015-11-10 06:53:58 [DEBUG   ] Got command DRX from MCS: ref #534654
2015-11-10 06:53:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:53:58 [DEBUG   ] run_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:53:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534654   8 57336 24838019 A  ERROR'
2015-11-10 06:53:58 [DEBUG   ] Got command DRX from MCS: ref #534653
2015-11-10 06:53:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:53:58 [DEBUG   ] run_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:53:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534653   8 57336 24838023 A  ERROR'
2015-11-10 06:53:58 [INFO    ] Finished starting DRX
2015-11-10 06:53:58 [INFO    ] Finished starting DRX
2015-11-10 06:53:58 [INFO    ] Finished starting DRX
2015-11-10 06:53:58 [INFO    ] Finished starting DRX
2015-11-10 06:53:58 [DEBUG   ] Got command BAM from MCS: ref #534656
2015-11-10 06:53:58 [INFO    ] Finished processing BAM request
2015-11-10 06:53:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534656   8 57336 24838139 A  ERROR'
2015-11-10 06:53:58 [DEBUG   ] Got command BAM from MCS: ref #534655
2015-11-10 06:53:58 [INFO    ] Finished processing BAM request
2015-11-10 06:53:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534655   8 57336 24838227 A  ERROR'
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 06:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 06:53:58 [DEBUG   ] spawn_process: at 2015-11-10 06:53:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 06:54:04 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-10 06:54:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 06:54:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 06:54:06 [DEBUG   ] Finished updating board flags in 1.170 seconds
2015-11-10 06:54:24 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:54:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:55:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 06:55:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 06:55:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:55:06 [DEBUG   ] Finished updating board flags in 1.035 seconds
2015-11-10 06:55:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 06:55:26 [DEBUG   ] Got command RPT from MCS: ref #534658
2015-11-10 06:55:26 [DEBUG   ] summary = ERROR
2015-11-10 06:55:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534658  13 57336 24926687 A  ERRORERROR'
2015-11-10 06:55:26 [DEBUG   ] Got command RPT from MCS: ref #534659
2015-11-10 06:55:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:55:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534659  77 57336 24926879 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:55:27 [DEBUG   ] Got command RPT from MCS: ref #534660
2015-11-10 06:55:27 [DEBUG   ] summary = ERROR
2015-11-10 06:55:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534660  13 57336 24927083 A  ERRORERROR'
2015-11-10 06:55:27 [DEBUG   ] Got command RPT from MCS: ref #534661
2015-11-10 06:55:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:55:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534661  77 57336 24927287 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:55:27 [DEBUG   ] Got command RPT from MCS: ref #534662
2015-11-10 06:55:27 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 06:55:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534662  12 57336 24927491 A  ERRORB–™š'
2015-11-10 06:55:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:55:58 [DEBUG   ] Got command DRX from MCS: ref #534668
2015-11-10 06:55:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:55:58 [DEBUG   ] run_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:55:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534668   8 57336 24958015 A  ERROR'
2015-11-10 06:55:58 [DEBUG   ] Got command DRX from MCS: ref #534669
2015-11-10 06:55:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:55:58 [DEBUG   ] run_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:55:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534669   8 57336 24958019 A  ERROR'
2015-11-10 06:55:58 [DEBUG   ] Got command DRX from MCS: ref #534670
2015-11-10 06:55:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:55:58 [DEBUG   ] run_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:55:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534670   8 57336 24958023 A  ERROR'
2015-11-10 06:55:58 [DEBUG   ] Got command DRX from MCS: ref #534671
2015-11-10 06:55:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:55:58 [DEBUG   ] run_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:55:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534671   8 57336 24958027 A  ERROR'
2015-11-10 06:55:58 [INFO    ] Finished starting DRX
2015-11-10 06:55:58 [INFO    ] Finished starting DRX
2015-11-10 06:55:58 [INFO    ] Finished starting DRX
2015-11-10 06:55:58 [INFO    ] Finished starting DRX
2015-11-10 06:55:58 [DEBUG   ] Got command BAM from MCS: ref #534672
2015-11-10 06:55:58 [INFO    ] Finished processing BAM request
2015-11-10 06:55:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534672   8 57336 24958211 A  ERROR'
2015-11-10 06:55:58 [DEBUG   ] Got command BAM from MCS: ref #534673
2015-11-10 06:55:58 [INFO    ] Finished processing BAM request
2015-11-10 06:55:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534673   8 57336 24958299 A  ERROR'
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 06:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 06:55:58 [DEBUG   ] spawn_process: at 2015-11-10 06:55:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 06:56:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 06:56:07 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:56:07 [DEBUG   ] Finished updating board flags in 1.992 seconds
2015-11-10 06:56:24 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:56:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:57:04 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 06:57:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 06:57:07 [DEBUG   ] Finished updating board flags in 1.788 seconds
2015-11-10 06:57:24 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2015-11-10 06:57:44 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:57:49 [DEBUG   ] Got command RPT from MCS: ref #534677
2015-11-10 06:57:49 [DEBUG   ] summary = ERROR
2015-11-10 06:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534677  13 57336 25069047 A  ERRORERROR'
2015-11-10 06:57:49 [DEBUG   ] Got command RPT from MCS: ref #534678
2015-11-10 06:57:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 06:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534678  77 57336 25069047 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 06:57:58 [DEBUG   ] Got command DRX from MCS: ref #534695
2015-11-10 06:57:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:57:58 [DEBUG   ] run_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:57:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534695   8 57336 25078015 A  ERROR'
2015-11-10 06:57:58 [DEBUG   ] Got command DRX from MCS: ref #534694
2015-11-10 06:57:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:57:58 [DEBUG   ] run_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:57:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534694   8 57336 25078019 A  ERROR'
2015-11-10 06:57:58 [DEBUG   ] Got command DRX from MCS: ref #534696
2015-11-10 06:57:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:57:58 [DEBUG   ] run_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:57:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534696   8 57336 25078023 A  ERROR'
2015-11-10 06:57:58 [DEBUG   ] Got command DRX from MCS: ref #534697
2015-11-10 06:57:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:57:58 [DEBUG   ] run_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:57:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534697   8 57336 25078027 A  ERROR'
2015-11-10 06:57:58 [INFO    ] Finished starting DRX
2015-11-10 06:57:58 [INFO    ] Finished starting DRX
2015-11-10 06:57:58 [INFO    ] Finished starting DRX
2015-11-10 06:57:58 [INFO    ] Finished starting DRX
2015-11-10 06:57:58 [DEBUG   ] Got command BAM from MCS: ref #534698
2015-11-10 06:57:58 [INFO    ] Finished processing BAM request
2015-11-10 06:57:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534698   8 57336 25078151 A  ERROR'
2015-11-10 06:57:58 [DEBUG   ] Got command BAM from MCS: ref #534699
2015-11-10 06:57:58 [INFO    ] Finished processing BAM request
2015-11-10 06:57:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534699   8 57336 25078263 A  ERROR'
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 06:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 06:57:58 [DEBUG   ] spawn_process: at 2015-11-10 06:57:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 06:58:04 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:58:07 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 06:58:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 06:58:07 [DEBUG   ] Finished updating board flags in 1.604 seconds
2015-11-10 06:58:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 06:58:44 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:59:04 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:59:07 [DEBUG   ] Finished updating board flags in 1.663 seconds
2015-11-10 06:59:24 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:59:44 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 06:59:58 [DEBUG   ] Got command DRX from MCS: ref #534703
2015-11-10 06:59:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:59:58 [DEBUG   ] run_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 06:59:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534703   8 57336 25198015 A  ERROR'
2015-11-10 06:59:58 [DEBUG   ] Got command DRX from MCS: ref #534704
2015-11-10 06:59:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:59:58 [DEBUG   ] run_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 06:59:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534704   8 57336 25198019 A  ERROR'
2015-11-10 06:59:58 [DEBUG   ] Got command DRX from MCS: ref #534705
2015-11-10 06:59:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:59:58 [DEBUG   ] run_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 06:59:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534705   8 57336 25198023 A  ERROR'
2015-11-10 06:59:58 [DEBUG   ] Got command DRX from MCS: ref #534706
2015-11-10 06:59:58 [INFO    ] sending drx command to lwa16
2015-11-10 06:59:58 [DEBUG   ] run_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 06:59:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 06:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534706   8 57336 25198027 A  ERROR'
2015-11-10 06:59:58 [INFO    ] Finished starting DRX
2015-11-10 06:59:58 [DEBUG   ] Got command BAM from MCS: ref #534707
2015-11-10 06:59:58 [INFO    ] Finished processing BAM request
2015-11-10 06:59:58 [INFO    ] Finished starting DRX
2015-11-10 06:59:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534707   8 57336 25198115 A  ERROR'
2015-11-10 06:59:58 [INFO    ] Finished starting DRX
2015-11-10 06:59:58 [INFO    ] Finished starting DRX
2015-11-10 06:59:58 [DEBUG   ] Got command BAM from MCS: ref #534708
2015-11-10 06:59:58 [INFO    ] Finished processing BAM request
2015-11-10 06:59:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 06:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534708   8 57336 25198251 A  ERROR'
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 06:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 06:59:58 [DEBUG   ] spawn_process: at 2015-11-10 06:59:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 07:00:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 07:00:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 07:00:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 07:00:06 [DEBUG   ] Finished updating board flags in 1.011 seconds
2015-11-10 07:00:24 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 07:00:26 [DEBUG   ] Got command RPT from MCS: ref #534709
2015-11-10 07:00:26 [DEBUG   ] summary = ERROR
2015-11-10 07:00:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534709  13 57336 25226759 A  ERRORERROR'
2015-11-10 07:00:26 [DEBUG   ] Got command RPT from MCS: ref #534710
2015-11-10 07:00:26 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:00:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534710  77 57336 25226963 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:00:27 [DEBUG   ] Got command RPT from MCS: ref #534711
2015-11-10 07:00:27 [DEBUG   ] summary = ERROR
2015-11-10 07:00:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534711  13 57336 25227167 A  ERRORERROR'
2015-11-10 07:00:27 [DEBUG   ] Got command RPT from MCS: ref #534712
2015-11-10 07:00:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:00:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534712  77 57336 25227371 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:00:27 [DEBUG   ] Got command RPT from MCS: ref #534713
2015-11-10 07:00:27 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 07:00:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534713  12 57336 25227563 A  ERRORB–™š'
2015-11-10 07:00:44 [DEBUG   ] Finished updating ant stats in 0.584 seconds
2015-11-10 07:01:04 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:01:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 07:01:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 07:01:06 [DEBUG   ] Finished updating board flags in 1.025 seconds
2015-11-10 07:01:24 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:01:44 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:01:58 [DEBUG   ] Got command DRX from MCS: ref #534714
2015-11-10 07:01:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:01:58 [DEBUG   ] run_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 07:01:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534714   8 57336 25318011 A  ERROR'
2015-11-10 07:01:58 [DEBUG   ] Got command DRX from MCS: ref #534715
2015-11-10 07:01:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:01:58 [DEBUG   ] run_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:01:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534715   8 57336 25318015 A  ERROR'
2015-11-10 07:01:58 [DEBUG   ] Got command DRX from MCS: ref #534716
2015-11-10 07:01:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:01:58 [DEBUG   ] run_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 07:01:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534716   8 57336 25318019 A  ERROR'
2015-11-10 07:01:58 [DEBUG   ] Got command DRX from MCS: ref #534717
2015-11-10 07:01:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:01:58 [DEBUG   ] run_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:01:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534717   8 57336 25318023 A  ERROR'
2015-11-10 07:01:58 [INFO    ] Finished starting DRX
2015-11-10 07:01:58 [INFO    ] Finished starting DRX
2015-11-10 07:01:58 [INFO    ] Finished starting DRX
2015-11-10 07:01:58 [INFO    ] Finished starting DRX
2015-11-10 07:01:58 [DEBUG   ] Got command BAM from MCS: ref #534718
2015-11-10 07:01:58 [INFO    ] Finished processing BAM request
2015-11-10 07:01:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534718   8 57336 25318187 A  ERROR'
2015-11-10 07:01:58 [DEBUG   ] Got command BAM from MCS: ref #534719
2015-11-10 07:01:58 [INFO    ] Finished processing BAM request
2015-11-10 07:01:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534719   8 57336 25318299 A  ERROR'
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 07:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 07:01:58 [DEBUG   ] spawn_process: at 2015-11-10 07:01:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 07:02:04 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:02:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 07:02:06 [DEBUG   ] Finished updating board flags in 0.963 seconds
2015-11-10 07:02:24 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:02:44 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:02:49 [DEBUG   ] Got command RPT from MCS: ref #534723
2015-11-10 07:02:49 [DEBUG   ] summary = ERROR
2015-11-10 07:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534723  13 57336 25369051 A  ERRORERROR'
2015-11-10 07:02:49 [DEBUG   ] Got command RPT from MCS: ref #534724
2015-11-10 07:02:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534724  77 57336 25369051 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:03:04 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:03:07 [DEBUG   ] Finished updating board flags in 1.370 seconds
2015-11-10 07:03:24 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:03:44 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:03:58 [DEBUG   ] Got command DRX from MCS: ref #534740
2015-11-10 07:03:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:03:58 [DEBUG   ] run_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 07:03:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534740   8 57336 25438011 A  ERROR'
2015-11-10 07:03:58 [DEBUG   ] Got command DRX from MCS: ref #534741
2015-11-10 07:03:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:03:58 [DEBUG   ] run_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:03:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534741   8 57336 25438015 A  ERROR'
2015-11-10 07:03:58 [DEBUG   ] Got command DRX from MCS: ref #534742
2015-11-10 07:03:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:03:58 [DEBUG   ] run_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 07:03:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534742   8 57336 25438019 A  ERROR'
2015-11-10 07:03:58 [DEBUG   ] Got command DRX from MCS: ref #534743
2015-11-10 07:03:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:03:58 [DEBUG   ] run_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:03:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534743   8 57336 25438023 A  ERROR'
2015-11-10 07:03:58 [INFO    ] Finished starting DRX
2015-11-10 07:03:58 [INFO    ] Finished starting DRX
2015-11-10 07:03:58 [INFO    ] Finished starting DRX
2015-11-10 07:03:58 [INFO    ] Finished starting DRX
2015-11-10 07:03:58 [DEBUG   ] Got command BAM from MCS: ref #534745
2015-11-10 07:03:58 [INFO    ] Finished processing BAM request
2015-11-10 07:03:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534745   8 57336 25438207 A  ERROR'
2015-11-10 07:03:58 [DEBUG   ] Got command BAM from MCS: ref #534744
2015-11-10 07:03:58 [INFO    ] Finished processing BAM request
2015-11-10 07:03:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534744   8 57336 25438343 A  ERROR'
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 07:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 07:03:58 [DEBUG   ] spawn_process: at 2015-11-10 07:03:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 07:04:04 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:04:07 [DEBUG   ] Finished updating board flags in 1.196 seconds
2015-11-10 07:04:24 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:04:44 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:05:04 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2015-11-10 07:05:07 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:05:07 [DEBUG   ] Finished updating board flags in 1.007 seconds
2015-11-10 07:05:24 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 07:05:26 [DEBUG   ] Got command RPT from MCS: ref #534747
2015-11-10 07:05:26 [DEBUG   ] summary = ERROR
2015-11-10 07:05:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534747  13 57336 25526859 A  ERRORERROR'
2015-11-10 07:05:27 [DEBUG   ] Got command RPT from MCS: ref #534748
2015-11-10 07:05:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:05:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534748  77 57336 25527063 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:05:27 [DEBUG   ] Got command RPT from MCS: ref #534749
2015-11-10 07:05:27 [DEBUG   ] summary = ERROR
2015-11-10 07:05:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534749  13 57336 25527255 A  ERRORERROR'
2015-11-10 07:05:27 [DEBUG   ] Got command RPT from MCS: ref #534750
2015-11-10 07:05:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:05:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534750  77 57336 25527459 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:05:27 [DEBUG   ] Got command RPT from MCS: ref #534751
2015-11-10 07:05:27 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 07:05:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534751  12 57336 25527663 A  ERRORB”™š'
2015-11-10 07:05:44 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:05:58 [DEBUG   ] Got command DRX from MCS: ref #534761
2015-11-10 07:05:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:05:58 [DEBUG   ] run_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:05:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534761   8 57336 25558011 A  ERROR'
2015-11-10 07:05:58 [DEBUG   ] Got command DRX from MCS: ref #534760
2015-11-10 07:05:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:05:58 [DEBUG   ] run_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 07:05:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534760   8 57336 25558015 A  ERROR'
2015-11-10 07:05:58 [DEBUG   ] Got command DRX from MCS: ref #534763
2015-11-10 07:05:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:05:58 [DEBUG   ] run_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:05:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534763   8 57336 25558019 A  ERROR'
2015-11-10 07:05:58 [DEBUG   ] Got command DRX from MCS: ref #534762
2015-11-10 07:05:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:05:58 [DEBUG   ] run_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 07:05:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534762   8 57336 25558023 A  ERROR'
2015-11-10 07:05:58 [INFO    ] Finished starting DRX
2015-11-10 07:05:58 [DEBUG   ] Got command BAM from MCS: ref #534765
2015-11-10 07:05:58 [INFO    ] Finished processing BAM request
2015-11-10 07:05:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534765   8 57336 25558099 A  ERROR'
2015-11-10 07:05:58 [INFO    ] Finished starting DRX
2015-11-10 07:05:58 [INFO    ] Finished starting DRX
2015-11-10 07:05:58 [INFO    ] Finished starting DRX
2015-11-10 07:05:58 [DEBUG   ] Got command BAM from MCS: ref #534764
2015-11-10 07:05:58 [INFO    ] Finished processing BAM request
2015-11-10 07:05:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534764   8 57336 25558223 A  ERROR'
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 07:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 07:05:58 [DEBUG   ] spawn_process: at 2015-11-10 07:05:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 07:06:04 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 07:06:07 [DEBUG   ] Finished updating board flags in 1.062 seconds
2015-11-10 07:06:24 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:06:44 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:07:04 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:07:08 [DEBUG   ] Finished updating board flags in 2.002 seconds
2015-11-10 07:07:24 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:07:44 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:07:49 [DEBUG   ] Got command RPT from MCS: ref #534766
2015-11-10 07:07:49 [DEBUG   ] summary = ERROR
2015-11-10 07:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534766  13 57336 25669047 A  ERRORERROR'
2015-11-10 07:07:49 [DEBUG   ] Got command RPT from MCS: ref #534767
2015-11-10 07:07:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534767  77 57336 25669047 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:07:58 [DEBUG   ] Got command DRX from MCS: ref #534783
2015-11-10 07:07:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:07:58 [DEBUG   ] run_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 07:07:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534783   8 57336 25678011 A  ERROR'
2015-11-10 07:07:58 [DEBUG   ] Got command DRX from MCS: ref #534784
2015-11-10 07:07:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:07:58 [DEBUG   ] run_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:07:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534784   8 57336 25678015 A  ERROR'
2015-11-10 07:07:58 [DEBUG   ] Got command DRX from MCS: ref #534785
2015-11-10 07:07:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:07:58 [DEBUG   ] run_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 07:07:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534785   8 57336 25678019 A  ERROR'
2015-11-10 07:07:58 [DEBUG   ] Got command DRX from MCS: ref #534786
2015-11-10 07:07:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:07:58 [DEBUG   ] run_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:07:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534786   8 57336 25678023 A  ERROR'
2015-11-10 07:07:58 [INFO    ] Finished starting DRX
2015-11-10 07:07:58 [INFO    ] Finished starting DRX
2015-11-10 07:07:58 [INFO    ] Finished starting DRX
2015-11-10 07:07:58 [INFO    ] Finished starting DRX
2015-11-10 07:07:58 [DEBUG   ] Got command BAM from MCS: ref #534788
2015-11-10 07:07:58 [INFO    ] Finished processing BAM request
2015-11-10 07:07:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534788   8 57336 25678127 A  ERROR'
2015-11-10 07:07:58 [DEBUG   ] Got command BAM from MCS: ref #534787
2015-11-10 07:07:58 [INFO    ] Finished processing BAM request
2015-11-10 07:07:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534787   8 57336 25678251 A  ERROR'
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 07:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 07:07:58 [DEBUG   ] spawn_process: at 2015-11-10 07:07:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 07:08:04 [DEBUG   ] Finished updating ant stats in 0.397 seconds
2015-11-10 07:08:07 [DEBUG   ] Finished updating board flags in 1.365 seconds
2015-11-10 07:08:25 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 07:08:45 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 07:09:05 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 07:09:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 07:09:07 [DEBUG   ] Finished updating board flags in 1.230 seconds
2015-11-10 07:09:25 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 07:09:45 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 07:09:58 [DEBUG   ] Got command DRX from MCS: ref #534792
2015-11-10 07:09:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:09:58 [DEBUG   ] run_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 07:09:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534792   8 57336 25798007 A  ERROR'
2015-11-10 07:09:58 [DEBUG   ] Got command DRX from MCS: ref #534793
2015-11-10 07:09:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:09:58 [DEBUG   ] run_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:09:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534793   8 57336 25798011 A  ERROR'
2015-11-10 07:09:58 [INFO    ] Finished starting DRX
2015-11-10 07:09:58 [INFO    ] Finished starting DRX
2015-11-10 07:09:58 [DEBUG   ] Got command DRX from MCS: ref #534794
2015-11-10 07:09:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:09:58 [DEBUG   ] run_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 07:09:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534794   8 57336 25798159 A  ERROR'
2015-11-10 07:09:58 [INFO    ] Finished starting DRX
2015-11-10 07:09:58 [DEBUG   ] Got command DRX from MCS: ref #534795
2015-11-10 07:09:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:09:58 [DEBUG   ] run_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:09:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534795   8 57336 25798295 A  ERROR'
2015-11-10 07:09:58 [INFO    ] Finished starting DRX
2015-11-10 07:09:58 [DEBUG   ] Got command BAM from MCS: ref #534796
2015-11-10 07:09:58 [INFO    ] Finished processing BAM request
2015-11-10 07:09:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534796   8 57336 25798419 A  ERROR'
2015-11-10 07:09:58 [DEBUG   ] Got command BAM from MCS: ref #534797
2015-11-10 07:09:58 [INFO    ] Finished processing BAM request
2015-11-10 07:09:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534797   8 57336 25798543 A  ERROR'
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 07:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 07:09:58 [DEBUG   ] spawn_process: at 2015-11-10 07:09:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 07:10:05 [DEBUG   ] Finished updating ant stats in 1.210 seconds
2015-11-10 07:10:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa34, chip 1
2015-11-10 07:10:09 [DEBUG   ] Finished updating board flags in 2.769 seconds
2015-11-10 07:10:25 [DEBUG   ] Finished updating ant stats in 1.193 seconds
2015-11-10 07:10:26 [DEBUG   ] Got command RPT from MCS: ref #534798
2015-11-10 07:10:26 [DEBUG   ] summary = ERROR
2015-11-10 07:10:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534798  13 57336 25826959 A  ERRORERROR'
2015-11-10 07:10:27 [DEBUG   ] Got command RPT from MCS: ref #534799
2015-11-10 07:10:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:10:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534799  77 57336 25827151 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:10:27 [DEBUG   ] Got command RPT from MCS: ref #534800
2015-11-10 07:10:27 [DEBUG   ] summary = ERROR
2015-11-10 07:10:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534800  13 57336 25827355 A  ERRORERROR'
2015-11-10 07:10:27 [DEBUG   ] Got command RPT from MCS: ref #534801
2015-11-10 07:10:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:10:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534801  77 57336 25827559 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:10:27 [DEBUG   ] Got command RPT from MCS: ref #534802
2015-11-10 07:10:27 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 07:10:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534802  12 57336 25827751 A  ERRORB–™š'
2015-11-10 07:10:45 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:11:05 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:11:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 07:11:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 07:11:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 07:11:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:11:07 [DEBUG   ] Finished updating board flags in 1.013 seconds
2015-11-10 07:11:25 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:11:45 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:11:58 [DEBUG   ] Got command DRX from MCS: ref #534807
2015-11-10 07:11:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:11:58 [DEBUG   ] run_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:11:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534807   8 57336 25918015 A  ERROR'
2015-11-10 07:11:58 [DEBUG   ] Got command DRX from MCS: ref #534806
2015-11-10 07:11:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:11:58 [DEBUG   ] run_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 07:11:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534806   8 57336 25918019 A  ERROR'
2015-11-10 07:11:58 [DEBUG   ] Got command DRX from MCS: ref #534808
2015-11-10 07:11:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:11:58 [DEBUG   ] run_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 07:11:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534808   8 57336 25918023 A  ERROR'
2015-11-10 07:11:58 [DEBUG   ] Got command DRX from MCS: ref #534809
2015-11-10 07:11:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:11:58 [DEBUG   ] run_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:11:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534809   8 57336 25918027 A  ERROR'
2015-11-10 07:11:58 [INFO    ] Finished starting DRX
2015-11-10 07:11:58 [DEBUG   ] Got command BAM from MCS: ref #534810
2015-11-10 07:11:58 [INFO    ] Finished processing BAM request
2015-11-10 07:11:58 [INFO    ] Finished starting DRX
2015-11-10 07:11:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534810   8 57336 25918127 A  ERROR'
2015-11-10 07:11:58 [INFO    ] Finished starting DRX
2015-11-10 07:11:58 [INFO    ] Finished starting DRX
2015-11-10 07:11:58 [DEBUG   ] Got command BAM from MCS: ref #534811
2015-11-10 07:11:58 [INFO    ] Finished processing BAM request
2015-11-10 07:11:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534811   8 57336 25918251 A  ERROR'
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 07:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 07:11:58 [DEBUG   ] spawn_process: at 2015-11-10 07:11:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 07:12:05 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:12:07 [DEBUG   ] Finished updating board flags in 1.016 seconds
2015-11-10 07:12:25 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:12:45 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:12:49 [DEBUG   ] Got command RPT from MCS: ref #534812
2015-11-10 07:12:49 [DEBUG   ] summary = ERROR
2015-11-10 07:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534812  13 57336 25969047 A  ERRORERROR'
2015-11-10 07:12:49 [DEBUG   ] Got command RPT from MCS: ref #534813
2015-11-10 07:12:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534813  77 57336 25969047 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:13:05 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:13:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 07:13:07 [DEBUG   ] Finished updating board flags in 1.018 seconds
2015-11-10 07:13:25 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:13:45 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:13:58 [DEBUG   ] Got command DRX from MCS: ref #534829
2015-11-10 07:13:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:13:58 [DEBUG   ] run_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 07:13:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534829   8 57336 26038011 A  ERROR'
2015-11-10 07:13:58 [DEBUG   ] Got command DRX from MCS: ref #534830
2015-11-10 07:13:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:13:58 [DEBUG   ] run_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:13:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534830   8 57336 26038015 A  ERROR'
2015-11-10 07:13:58 [DEBUG   ] Got command DRX from MCS: ref #534831
2015-11-10 07:13:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:13:58 [DEBUG   ] run_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 07:13:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534831   8 57336 26038019 A  ERROR'
2015-11-10 07:13:58 [DEBUG   ] Got command DRX from MCS: ref #534832
2015-11-10 07:13:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:13:58 [DEBUG   ] run_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:13:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534832   8 57336 26038023 A  ERROR'
2015-11-10 07:13:58 [INFO    ] Finished starting DRX
2015-11-10 07:13:58 [INFO    ] Finished starting DRX
2015-11-10 07:13:58 [INFO    ] Finished starting DRX
2015-11-10 07:13:58 [INFO    ] Finished starting DRX
2015-11-10 07:13:58 [DEBUG   ] Got command BAM from MCS: ref #534833
2015-11-10 07:13:58 [INFO    ] Finished processing BAM request
2015-11-10 07:13:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534833   8 57336 26038187 A  ERROR'
2015-11-10 07:13:58 [DEBUG   ] Got command BAM from MCS: ref #534834
2015-11-10 07:13:58 [INFO    ] Finished processing BAM request
2015-11-10 07:13:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534834   8 57336 26038275 A  ERROR'
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 07:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 07:13:58 [DEBUG   ] spawn_process: at 2015-11-10 07:13:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 07:14:05 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:14:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 07:14:08 [DEBUG   ] Finished updating board flags in 1.541 seconds
2015-11-10 07:14:25 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:14:45 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:15:05 [DEBUG   ] Finished updating ant stats in 1.064 seconds
2015-11-10 07:15:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 07:15:08 [DEBUG   ] Finished updating board flags in 1.386 seconds
2015-11-10 07:15:25 [DEBUG   ] Finished updating ant stats in 0.936 seconds
2015-11-10 07:15:27 [DEBUG   ] Got command RPT from MCS: ref #534839
2015-11-10 07:15:27 [DEBUG   ] summary = ERROR
2015-11-10 07:15:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534839  13 57336 26127055 A  ERRORERROR'
2015-11-10 07:15:27 [DEBUG   ] Got command RPT from MCS: ref #534840
2015-11-10 07:15:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:15:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534840  77 57336 26127259 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:15:27 [DEBUG   ] Got command RPT from MCS: ref #534841
2015-11-10 07:15:27 [DEBUG   ] summary = ERROR
2015-11-10 07:15:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534841  13 57336 26127463 A  ERRORERROR'
2015-11-10 07:15:27 [DEBUG   ] Got command RPT from MCS: ref #534842
2015-11-10 07:15:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:15:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534842  77 57336 26127667 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:15:27 [DEBUG   ] Got command RPT from MCS: ref #534843
2015-11-10 07:15:27 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 07:15:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534843  12 57336 26127859 A  ERRORB–™š'
2015-11-10 07:15:45 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:15:58 [DEBUG   ] Got command DRX from MCS: ref #534850
2015-11-10 07:15:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:15:58 [DEBUG   ] run_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:15:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534850   8 57336 26158007 A  ERROR'
2015-11-10 07:15:58 [DEBUG   ] Got command DRX from MCS: ref #534849
2015-11-10 07:15:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:15:58 [DEBUG   ] run_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 07:15:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534849   8 57336 26158011 A  ERROR'
2015-11-10 07:15:58 [INFO    ] Finished starting DRX
2015-11-10 07:15:58 [INFO    ] Finished starting DRX
2015-11-10 07:15:58 [DEBUG   ] Got command DRX from MCS: ref #534851
2015-11-10 07:15:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:15:58 [DEBUG   ] run_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 07:15:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534851   8 57336 26158147 A  ERROR'
2015-11-10 07:15:58 [INFO    ] Finished starting DRX
2015-11-10 07:15:58 [DEBUG   ] Got command DRX from MCS: ref #534852
2015-11-10 07:15:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:15:58 [DEBUG   ] run_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:15:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534852   8 57336 26158259 A  ERROR'
2015-11-10 07:15:58 [INFO    ] Finished starting DRX
2015-11-10 07:15:58 [DEBUG   ] Got command BAM from MCS: ref #534854
2015-11-10 07:15:58 [INFO    ] Finished processing BAM request
2015-11-10 07:15:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534854   8 57336 26158395 A  ERROR'
2015-11-10 07:15:58 [DEBUG   ] Got command BAM from MCS: ref #534853
2015-11-10 07:15:58 [INFO    ] Finished processing BAM request
2015-11-10 07:15:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534853   8 57336 26158519 A  ERROR'
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 07:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 07:15:58 [DEBUG   ] spawn_process: at 2015-11-10 07:15:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 07:16:05 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:16:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 07:16:08 [DEBUG   ] Finished updating board flags in 1.199 seconds
2015-11-10 07:16:25 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:16:45 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:17:05 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:17:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2015-11-10 07:17:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 07:17:08 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:17:08 [DEBUG   ] Finished updating board flags in 1.087 seconds
2015-11-10 07:17:25 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:17:45 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:17:49 [DEBUG   ] Got command RPT from MCS: ref #534858
2015-11-10 07:17:49 [DEBUG   ] summary = ERROR
2015-11-10 07:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534858  13 57336 26269055 A  ERRORERROR'
2015-11-10 07:17:49 [DEBUG   ] Got command RPT from MCS: ref #534859
2015-11-10 07:17:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534859  77 57336 26269055 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:17:58 [DEBUG   ] Got command DRX from MCS: ref #534875
2015-11-10 07:17:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:17:58 [DEBUG   ] run_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 07:17:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534875   8 57336 26278011 A  ERROR'
2015-11-10 07:17:58 [DEBUG   ] Got command DRX from MCS: ref #534876
2015-11-10 07:17:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:17:58 [DEBUG   ] run_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:17:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534876   8 57336 26278015 A  ERROR'
2015-11-10 07:17:58 [DEBUG   ] Got command DRX from MCS: ref #534877
2015-11-10 07:17:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:17:58 [DEBUG   ] run_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 07:17:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534877   8 57336 26278019 A  ERROR'
2015-11-10 07:17:58 [DEBUG   ] Got command DRX from MCS: ref #534878
2015-11-10 07:17:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:17:58 [DEBUG   ] run_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:17:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534878   8 57336 26278023 A  ERROR'
2015-11-10 07:17:58 [INFO    ] Finished starting DRX
2015-11-10 07:17:58 [INFO    ] Finished starting DRX
2015-11-10 07:17:58 [INFO    ] Finished starting DRX
2015-11-10 07:17:58 [INFO    ] Finished starting DRX
2015-11-10 07:17:58 [DEBUG   ] Got command BAM from MCS: ref #534880
2015-11-10 07:17:58 [INFO    ] Finished processing BAM request
2015-11-10 07:17:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534880   8 57336 26278219 A  ERROR'
2015-11-10 07:17:58 [DEBUG   ] Got command BAM from MCS: ref #534879
2015-11-10 07:17:58 [INFO    ] Finished processing BAM request
2015-11-10 07:17:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534879   8 57336 26278331 A  ERROR'
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 07:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 07:17:58 [DEBUG   ] spawn_process: at 2015-11-10 07:17:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 07:18:05 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:18:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 07:18:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 07:18:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:18:09 [DEBUG   ] Finished updating board flags in 2.004 seconds
2015-11-10 07:18:25 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:18:45 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:19:05 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:19:08 [DEBUG   ] Finished updating board flags in 1.364 seconds
2015-11-10 07:19:25 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:19:45 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:19:58 [DEBUG   ] Got command DRX from MCS: ref #534881
2015-11-10 07:19:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:19:58 [DEBUG   ] run_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 07:19:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534881   8 57336 26398007 A  ERROR'
2015-11-10 07:19:58 [DEBUG   ] Got command DRX from MCS: ref #534882
2015-11-10 07:19:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:19:58 [DEBUG   ] run_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:19:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534882   8 57336 26398011 A  ERROR'
2015-11-10 07:19:58 [INFO    ] Finished starting DRX
2015-11-10 07:19:58 [INFO    ] Finished starting DRX
2015-11-10 07:19:58 [DEBUG   ] Got command DRX from MCS: ref #534883
2015-11-10 07:19:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:19:58 [DEBUG   ] run_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 07:19:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534883   8 57336 26398171 A  ERROR'
2015-11-10 07:19:58 [INFO    ] Finished starting DRX
2015-11-10 07:19:58 [DEBUG   ] Got command DRX from MCS: ref #534884
2015-11-10 07:19:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:19:58 [DEBUG   ] run_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:19:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534884   8 57336 26398287 A  ERROR'
2015-11-10 07:19:58 [INFO    ] Finished starting DRX
2015-11-10 07:19:58 [DEBUG   ] Got command BAM from MCS: ref #534886
2015-11-10 07:19:58 [INFO    ] Finished processing BAM request
2015-11-10 07:19:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534886   8 57336 26398399 A  ERROR'
2015-11-10 07:19:58 [DEBUG   ] Got command BAM from MCS: ref #534885
2015-11-10 07:19:58 [INFO    ] Finished processing BAM request
2015-11-10 07:19:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:19:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534885   8 57336 26398535 A  ERROR'
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 07:19:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 07:19:58 [DEBUG   ] spawn_process: at 2015-11-10 07:19:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 07:20:05 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:20:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 07:20:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 07:20:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2015-11-10 07:20:08 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:20:08 [DEBUG   ] Finished updating board flags in 1.274 seconds
2015-11-10 07:20:25 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:20:27 [DEBUG   ] Got command RPT from MCS: ref #534887
2015-11-10 07:20:27 [DEBUG   ] summary = ERROR
2015-11-10 07:20:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534887  13 57336 26427159 A  ERRORERROR'
2015-11-10 07:20:27 [DEBUG   ] Got command RPT from MCS: ref #534888
2015-11-10 07:20:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:20:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534888  77 57336 26427363 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:20:27 [DEBUG   ] Got command RPT from MCS: ref #534889
2015-11-10 07:20:27 [DEBUG   ] summary = ERROR
2015-11-10 07:20:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534889  13 57336 26427567 A  ERRORERROR'
2015-11-10 07:20:27 [DEBUG   ] Got command RPT from MCS: ref #534890
2015-11-10 07:20:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:20:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534890  77 57336 26427771 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:20:27 [DEBUG   ] Got command RPT from MCS: ref #534891
2015-11-10 07:20:27 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 07:20:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534891  12 57336 26427963 A  ERRORB—™š'
2015-11-10 07:20:45 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:21:05 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:21:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:21:08 [DEBUG   ] Finished updating board flags in 1.170 seconds
2015-11-10 07:21:25 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:21:45 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:21:58 [DEBUG   ] Got command DRX from MCS: ref #534896
2015-11-10 07:21:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:21:58 [DEBUG   ] run_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:21:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534896   8 57336 26518015 A  ERROR'
2015-11-10 07:21:58 [DEBUG   ] Got command DRX from MCS: ref #534895
2015-11-10 07:21:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:21:58 [DEBUG   ] run_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 07:21:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534895   8 57336 26518019 A  ERROR'
2015-11-10 07:21:58 [DEBUG   ] Got command DRX from MCS: ref #534897
2015-11-10 07:21:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:21:58 [DEBUG   ] run_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 07:21:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534897   8 57336 26518023 A  ERROR'
2015-11-10 07:21:58 [DEBUG   ] Got command DRX from MCS: ref #534898
2015-11-10 07:21:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:21:58 [DEBUG   ] run_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:21:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534898   8 57336 26518027 A  ERROR'
2015-11-10 07:21:58 [INFO    ] Finished starting DRX
2015-11-10 07:21:58 [INFO    ] Finished starting DRX
2015-11-10 07:21:58 [INFO    ] Finished starting DRX
2015-11-10 07:21:58 [INFO    ] Finished starting DRX
2015-11-10 07:21:58 [DEBUG   ] Got command BAM from MCS: ref #534899
2015-11-10 07:21:58 [INFO    ] Finished processing BAM request
2015-11-10 07:21:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534899   8 57336 26518223 A  ERROR'
2015-11-10 07:21:58 [DEBUG   ] Got command BAM from MCS: ref #534900
2015-11-10 07:21:58 [INFO    ] Finished processing BAM request
2015-11-10 07:21:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:21:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534900   8 57336 26518359 A  ERROR'
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 07:21:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 07:21:58 [DEBUG   ] spawn_process: at 2015-11-10 07:21:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 07:22:05 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 07:22:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:22:08 [DEBUG   ] Finished updating board flags in 0.969 seconds
2015-11-10 07:22:25 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2015-11-10 07:22:45 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 07:22:49 [DEBUG   ] Got command RPT from MCS: ref #534901
2015-11-10 07:22:49 [DEBUG   ] summary = ERROR
2015-11-10 07:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534901  13 57336 26569047 A  ERRORERROR'
2015-11-10 07:22:49 [DEBUG   ] Got command RPT from MCS: ref #534902
2015-11-10 07:22:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534902  77 57336 26569047 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:23:05 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 07:23:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 07:23:09 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:23:09 [DEBUG   ] Finished updating board flags in 1.364 seconds
2015-11-10 07:23:25 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 07:23:45 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 07:23:58 [DEBUG   ] Got command DRX from MCS: ref #534921
2015-11-10 07:23:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:23:58 [DEBUG   ] run_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 07:23:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534921   8 57336 26638007 A  ERROR'
2015-11-10 07:23:58 [DEBUG   ] Got command DRX from MCS: ref #534922
2015-11-10 07:23:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:23:58 [DEBUG   ] run_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:23:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534922   8 57336 26638011 A  ERROR'
2015-11-10 07:23:58 [INFO    ] Finished starting DRX
2015-11-10 07:23:58 [INFO    ] Finished starting DRX
2015-11-10 07:23:58 [DEBUG   ] Got command DRX from MCS: ref #534923
2015-11-10 07:23:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:23:58 [DEBUG   ] run_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 07:23:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534923   8 57336 26638159 A  ERROR'
2015-11-10 07:23:58 [INFO    ] Finished starting DRX
2015-11-10 07:23:58 [DEBUG   ] Got command DRX from MCS: ref #534924
2015-11-10 07:23:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:23:58 [DEBUG   ] run_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:23:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534924   8 57336 26638283 A  ERROR'
2015-11-10 07:23:58 [INFO    ] Finished starting DRX
2015-11-10 07:23:58 [DEBUG   ] Got command BAM from MCS: ref #534925
2015-11-10 07:23:58 [INFO    ] Finished processing BAM request
2015-11-10 07:23:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534925   8 57336 26638407 A  ERROR'
2015-11-10 07:23:58 [DEBUG   ] Got command BAM from MCS: ref #534926
2015-11-10 07:23:58 [INFO    ] Finished processing BAM request
2015-11-10 07:23:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:23:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534926   8 57336 26638519 A  ERROR'
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 07:23:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 07:23:58 [DEBUG   ] spawn_process: at 2015-11-10 07:23:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 07:24:05 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 07:24:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 07:24:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:24:09 [DEBUG   ] Finished updating board flags in 1.237 seconds
2015-11-10 07:24:25 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 07:24:45 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 07:25:05 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 07:25:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 07:25:09 [DEBUG   ] Finished updating board flags in 1.233 seconds
2015-11-10 07:25:25 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2015-11-10 07:25:27 [DEBUG   ] Got command RPT from MCS: ref #534928
2015-11-10 07:25:27 [DEBUG   ] summary = ERROR
2015-11-10 07:25:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534928  13 57336 26727247 A  ERRORERROR'
2015-11-10 07:25:27 [DEBUG   ] Got command RPT from MCS: ref #534929
2015-11-10 07:25:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:25:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534929  77 57336 26727451 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:25:27 [DEBUG   ] Got command RPT from MCS: ref #534930
2015-11-10 07:25:27 [DEBUG   ] summary = ERROR
2015-11-10 07:25:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534930  13 57336 26727655 A  ERRORERROR'
2015-11-10 07:25:27 [DEBUG   ] Got command RPT from MCS: ref #534931
2015-11-10 07:25:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:25:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534931  77 57336 26727847 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:25:28 [DEBUG   ] Got command RPT from MCS: ref #534932
2015-11-10 07:25:28 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 07:25:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534932  12 57336 26728051 A  ERRORB•™š'
2015-11-10 07:25:45 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 07:25:58 [DEBUG   ] Got command DRX from MCS: ref #534938
2015-11-10 07:25:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:25:58 [DEBUG   ] run_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 07:25:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534938   8 57336 26758007 A  ERROR'
2015-11-10 07:25:58 [DEBUG   ] Got command DRX from MCS: ref #534939
2015-11-10 07:25:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:25:58 [DEBUG   ] run_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:25:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534939   8 57336 26758011 A  ERROR'
2015-11-10 07:25:58 [INFO    ] Finished starting DRX
2015-11-10 07:25:58 [INFO    ] Finished starting DRX
2015-11-10 07:25:58 [DEBUG   ] Got command DRX from MCS: ref #534940
2015-11-10 07:25:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:25:58 [DEBUG   ] run_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 07:25:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534940   8 57336 26758183 A  ERROR'
2015-11-10 07:25:58 [INFO    ] Finished starting DRX
2015-11-10 07:25:58 [DEBUG   ] Got command DRX from MCS: ref #534941
2015-11-10 07:25:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:25:58 [DEBUG   ] run_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:25:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534941   8 57336 26758331 A  ERROR'
2015-11-10 07:25:58 [INFO    ] Finished starting DRX
2015-11-10 07:25:58 [DEBUG   ] Got command BAM from MCS: ref #534942
2015-11-10 07:25:58 [INFO    ] Finished processing BAM request
2015-11-10 07:25:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534942   8 57336 26758431 A  ERROR'
2015-11-10 07:25:58 [DEBUG   ] Got command BAM from MCS: ref #534943
2015-11-10 07:25:58 [INFO    ] Finished processing BAM request
2015-11-10 07:25:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:25:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534943   8 57336 26758531 A  ERROR'
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 07:25:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 07:25:58 [DEBUG   ] spawn_process: at 2015-11-10 07:25:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 07:26:05 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:26:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 07:26:09 [DEBUG   ] Finished updating board flags in 1.122 seconds
2015-11-10 07:26:25 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:26:45 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:27:05 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:27:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:27:09 [DEBUG   ] Finished updating board flags in 1.026 seconds
2015-11-10 07:27:25 [DEBUG   ] Finished updating ant stats in 0.392 seconds
2015-11-10 07:27:46 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 07:27:49 [DEBUG   ] Got command RPT from MCS: ref #534947
2015-11-10 07:27:49 [DEBUG   ] summary = ERROR
2015-11-10 07:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534947  13 57336 26869055 A  ERRORERROR'
2015-11-10 07:27:49 [DEBUG   ] Got command RPT from MCS: ref #534948
2015-11-10 07:27:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534948  77 57336 26869055 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:27:58 [DEBUG   ] Got command DRX from MCS: ref #534965
2015-11-10 07:27:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:27:58 [DEBUG   ] run_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:27:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534965   8 57336 26878011 A  ERROR'
2015-11-10 07:27:58 [DEBUG   ] Got command DRX from MCS: ref #534964
2015-11-10 07:27:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:27:58 [DEBUG   ] run_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 36000000.0 7 9 0']
2015-11-10 07:27:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534964   8 57336 26878015 A  ERROR'
2015-11-10 07:27:58 [DEBUG   ] Got command DRX from MCS: ref #534966
2015-11-10 07:27:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:27:58 [DEBUG   ] run_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 54000000.0 7 9 0']
2015-11-10 07:27:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534966   8 57336 26878019 A  ERROR'
2015-11-10 07:27:58 [DEBUG   ] Got command DRX from MCS: ref #534967
2015-11-10 07:27:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:27:58 [DEBUG   ] run_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:27:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534967   8 57336 26878023 A  ERROR'
2015-11-10 07:27:58 [INFO    ] Finished starting DRX
2015-11-10 07:27:58 [INFO    ] Finished starting DRX
2015-11-10 07:27:58 [INFO    ] Finished starting DRX
2015-11-10 07:27:58 [INFO    ] Finished starting DRX
2015-11-10 07:27:58 [DEBUG   ] Got command BAM from MCS: ref #534969
2015-11-10 07:27:58 [INFO    ] Finished processing BAM request
2015-11-10 07:27:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534969   8 57336 26878171 A  ERROR'
2015-11-10 07:27:58 [DEBUG   ] Got command BAM from MCS: ref #534968
2015-11-10 07:27:58 [INFO    ] Finished processing BAM request
2015-11-10 07:27:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534968   8 57336 26878307 A  ERROR'
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 07:27:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 07:27:58 [DEBUG   ] spawn_process: at 2015-11-10 07:27:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 07:28:06 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 07:28:10 [DEBUG   ] Finished updating board flags in 2.007 seconds
2015-11-10 07:28:26 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 07:28:46 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 07:29:06 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 07:29:10 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:29:10 [DEBUG   ] Finished updating board flags in 1.968 seconds
2015-11-10 07:29:26 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 07:29:46 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 07:29:58 [DEBUG   ] Got command DRX from MCS: ref #534973
2015-11-10 07:29:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:29:58 [DEBUG   ] run_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:29:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534973   8 57336 26998007 A  ERROR'
2015-11-10 07:29:58 [INFO    ] Finished starting DRX
2015-11-10 07:29:58 [DEBUG   ] Got command DRX from MCS: ref #534974
2015-11-10 07:29:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:29:58 [DEBUG   ] run_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:29:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534974   8 57336 26998155 A  ERROR'
2015-11-10 07:29:58 [DEBUG   ] Got command BAM from MCS: ref #534975
2015-11-10 07:29:58 [INFO    ] Finished processing BAM request
2015-11-10 07:29:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:29:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534975   8 57336 26998159 A  ERROR'
2015-11-10 07:29:58 [INFO    ] Finished starting DRX
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 07:29:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 07:29:58 [DEBUG   ] spawn_process: at 2015-11-10 07:29:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 07:30:00 [DEBUG   ] Got command STP from MCS: ref #534976
2015-11-10 07:30:00 [INFO    ] Finished processing BAM request
2015-11-10 07:30:00 [INFO    ] Shutdown the adder chain on beam #3
2015-11-10 07:30:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   534976   8 57336 27000023 A  ERROR'
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 07:30:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 07:30:00 [DEBUG   ] spawn_process: at 2015-11-10 07:30:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 07:30:06 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 07:30:10 [DEBUG   ] Finished updating board flags in 1.815 seconds
2015-11-10 07:30:26 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2015-11-10 07:30:27 [DEBUG   ] Got command RPT from MCS: ref #534977
2015-11-10 07:30:27 [DEBUG   ] summary = ERROR
2015-11-10 07:30:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534977  13 57336 27027343 A  ERRORERROR'
2015-11-10 07:30:27 [DEBUG   ] Got command RPT from MCS: ref #534978
2015-11-10 07:30:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:30:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534978  77 57336 27027535 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:30:27 [DEBUG   ] Got command RPT from MCS: ref #534979
2015-11-10 07:30:27 [DEBUG   ] summary = ERROR
2015-11-10 07:30:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534979  13 57336 27027739 A  ERRORERROR'
2015-11-10 07:30:27 [DEBUG   ] Got command RPT from MCS: ref #534980
2015-11-10 07:30:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:30:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534980  77 57336 27027943 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:30:28 [DEBUG   ] Got command RPT from MCS: ref #534981
2015-11-10 07:30:28 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 07:30:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534981  12 57336 27028147 A  ERRORB–™š'
2015-11-10 07:30:46 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:31:06 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:31:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 07:31:09 [DEBUG   ] Finished updating board flags in 1.255 seconds
2015-11-10 07:31:26 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:31:46 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:31:58 [DEBUG   ] Got command DRX from MCS: ref #534982
2015-11-10 07:31:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:31:58 [DEBUG   ] run_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:31:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534982   8 57336 27118015 A  ERROR'
2015-11-10 07:31:58 [DEBUG   ] Got command DRX from MCS: ref #534983
2015-11-10 07:31:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:31:58 [DEBUG   ] run_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:31:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   534983   8 57336 27118019 A  ERROR'
2015-11-10 07:31:58 [INFO    ] Finished starting DRX
2015-11-10 07:31:58 [INFO    ] Finished starting DRX
2015-11-10 07:31:58 [DEBUG   ] Got command BAM from MCS: ref #534984
2015-11-10 07:31:58 [INFO    ] Finished processing BAM request
2015-11-10 07:31:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:31:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   534984   8 57336 27118227 A  ERROR'
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 07:31:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 07:31:58 [DEBUG   ] spawn_process: at 2015-11-10 07:31:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 07:32:06 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:32:09 [DEBUG   ] Finished updating board flags in 1.187 seconds
2015-11-10 07:32:26 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:32:46 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:32:49 [DEBUG   ] Got command RPT from MCS: ref #534988
2015-11-10 07:32:49 [DEBUG   ] summary = ERROR
2015-11-10 07:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534988  13 57336 27169051 A  ERRORERROR'
2015-11-10 07:32:49 [DEBUG   ] Got command RPT from MCS: ref #534989
2015-11-10 07:32:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   534989  77 57336 27169051 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:33:06 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:33:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:33:09 [DEBUG   ] Finished updating board flags in 1.035 seconds
2015-11-10 07:33:26 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:33:46 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:33:58 [DEBUG   ] Got command DRX from MCS: ref #535005
2015-11-10 07:33:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:33:58 [DEBUG   ] run_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:33:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535005   8 57336 27238007 A  ERROR'
2015-11-10 07:33:58 [INFO    ] Finished starting DRX
2015-11-10 07:33:58 [DEBUG   ] Got command DRX from MCS: ref #535006
2015-11-10 07:33:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:33:58 [DEBUG   ] run_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:33:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535006   8 57336 27238143 A  ERROR'
2015-11-10 07:33:58 [DEBUG   ] Got command BAM from MCS: ref #535007
2015-11-10 07:33:58 [INFO    ] Finished processing BAM request
2015-11-10 07:33:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:33:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535007   8 57336 27238147 A  ERROR'
2015-11-10 07:33:58 [INFO    ] Finished starting DRX
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 07:33:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 07:33:58 [DEBUG   ] spawn_process: at 2015-11-10 07:33:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 07:34:06 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:34:09 [DEBUG   ] Finished updating board flags in 1.050 seconds
2015-11-10 07:34:26 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:34:46 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:35:06 [DEBUG   ] Finished updating ant stats in 1.013 seconds
2015-11-10 07:35:09 [DEBUG   ] Finished updating board flags in 0.934 seconds
2015-11-10 07:35:26 [DEBUG   ] Finished updating ant stats in 0.988 seconds
2015-11-10 07:35:27 [DEBUG   ] Got command RPT from MCS: ref #535009
2015-11-10 07:35:27 [DEBUG   ] summary = ERROR
2015-11-10 07:35:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535009  13 57336 27327455 A  ERRORERROR'
2015-11-10 07:35:27 [DEBUG   ] Got command RPT from MCS: ref #535010
2015-11-10 07:35:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:35:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535010  77 57336 27327659 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:35:27 [DEBUG   ] Got command RPT from MCS: ref #535011
2015-11-10 07:35:27 [DEBUG   ] summary = ERROR
2015-11-10 07:35:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535011  13 57336 27327863 A  ERRORERROR'
2015-11-10 07:35:28 [DEBUG   ] Got command RPT from MCS: ref #535012
2015-11-10 07:35:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:35:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535012  77 57336 27328067 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:35:28 [DEBUG   ] Got command RPT from MCS: ref #535013
2015-11-10 07:35:28 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 07:35:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535013  12 57336 27328259 A  ERRORB‘™š'
2015-11-10 07:35:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:35:58 [DEBUG   ] Got command DRX from MCS: ref #535022
2015-11-10 07:35:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:35:58 [DEBUG   ] run_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:35:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535022   8 57336 27358015 A  ERROR'
2015-11-10 07:35:58 [DEBUG   ] Got command DRX from MCS: ref #535023
2015-11-10 07:35:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:35:58 [DEBUG   ] run_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:35:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535023   8 57336 27358019 A  ERROR'
2015-11-10 07:35:58 [INFO    ] Finished starting DRX
2015-11-10 07:35:58 [INFO    ] Finished starting DRX
2015-11-10 07:35:58 [DEBUG   ] Got command BAM from MCS: ref #535024
2015-11-10 07:35:58 [INFO    ] Finished processing BAM request
2015-11-10 07:35:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535024   8 57336 27358167 A  ERROR'
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 07:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 07:35:58 [DEBUG   ] spawn_process: at 2015-11-10 07:35:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 07:36:06 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 07:36:10 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:36:10 [DEBUG   ] Finished updating board flags in 1.341 seconds
2015-11-10 07:36:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 07:36:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 07:37:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 07:37:10 [DEBUG   ] Finished updating board flags in 1.200 seconds
2015-11-10 07:37:26 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 07:37:46 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 07:37:49 [DEBUG   ] Got command RPT from MCS: ref #535025
2015-11-10 07:37:49 [DEBUG   ] summary = ERROR
2015-11-10 07:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535025  13 57336 27469059 A  ERRORERROR'
2015-11-10 07:37:49 [DEBUG   ] Got command RPT from MCS: ref #535026
2015-11-10 07:37:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535026  77 57336 27469059 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:37:58 [DEBUG   ] Got command DRX from MCS: ref #535042
2015-11-10 07:37:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:37:58 [DEBUG   ] run_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:37:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535042   8 57336 27478015 A  ERROR'
2015-11-10 07:37:58 [DEBUG   ] Got command DRX from MCS: ref #535043
2015-11-10 07:37:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:37:58 [DEBUG   ] run_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:37:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535043   8 57336 27478019 A  ERROR'
2015-11-10 07:37:58 [INFO    ] Finished starting DRX
2015-11-10 07:37:58 [INFO    ] Finished starting DRX
2015-11-10 07:37:58 [DEBUG   ] Got command BAM from MCS: ref #535044
2015-11-10 07:37:58 [INFO    ] Finished processing BAM request
2015-11-10 07:37:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535044   8 57336 27478131 A  ERROR'
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 07:37:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 07:37:58 [DEBUG   ] spawn_process: at 2015-11-10 07:37:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 07:38:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 07:38:10 [DEBUG   ] Finished updating board flags in 1.223 seconds
2015-11-10 07:38:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 07:38:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 07:39:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 07:39:10 [DEBUG   ] Finished updating board flags in 1.213 seconds
2015-11-10 07:39:26 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 07:39:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 07:39:58 [DEBUG   ] Got command DRX from MCS: ref #535048
2015-11-10 07:39:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:39:58 [DEBUG   ] run_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535048   8 57336 27598015 A  ERROR'
2015-11-10 07:39:58 [DEBUG   ] Got command DRX from MCS: ref #535049
2015-11-10 07:39:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:39:58 [DEBUG   ] run_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535049   8 57336 27598019 A  ERROR'
2015-11-10 07:39:58 [INFO    ] Finished starting DRX
2015-11-10 07:39:58 [INFO    ] Finished starting DRX
2015-11-10 07:39:58 [DEBUG   ] Got command BAM from MCS: ref #535050
2015-11-10 07:39:58 [INFO    ] Finished processing BAM request
2015-11-10 07:39:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535050   8 57336 27598167 A  ERROR'
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 07:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 07:39:58 [DEBUG   ] spawn_process: at 2015-11-10 07:39:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 07:40:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 07:40:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 07:40:10 [DEBUG   ] Finished updating board flags in 1.257 seconds
2015-11-10 07:40:26 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2015-11-10 07:40:27 [DEBUG   ] Got command RPT from MCS: ref #535051
2015-11-10 07:40:27 [DEBUG   ] summary = ERROR
2015-11-10 07:40:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535051  13 57336 27627555 A  ERRORERROR'
2015-11-10 07:40:27 [DEBUG   ] Got command RPT from MCS: ref #535052
2015-11-10 07:40:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:40:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535052  77 57336 27627759 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:40:27 [DEBUG   ] Got command RPT from MCS: ref #535053
2015-11-10 07:40:27 [DEBUG   ] summary = ERROR
2015-11-10 07:40:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535053  13 57336 27627951 A  ERRORERROR'
2015-11-10 07:40:28 [DEBUG   ] Got command RPT from MCS: ref #535054
2015-11-10 07:40:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:40:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535054  77 57336 27628155 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:40:28 [DEBUG   ] Got command RPT from MCS: ref #535055
2015-11-10 07:40:28 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 07:40:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535055  12 57336 27628359 A  ERRORB‘™š'
2015-11-10 07:40:46 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:41:06 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:41:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 07:41:10 [DEBUG   ] Finished updating board flags in 1.112 seconds
2015-11-10 07:41:26 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 07:41:46 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 07:41:58 [DEBUG   ] Got command DRX from MCS: ref #535059
2015-11-10 07:41:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:41:58 [DEBUG   ] run_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:41:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535059   8 57336 27718011 A  ERROR'
2015-11-10 07:41:58 [DEBUG   ] Got command DRX from MCS: ref #535060
2015-11-10 07:41:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:41:58 [DEBUG   ] run_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:41:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535060   8 57336 27718015 A  ERROR'
2015-11-10 07:41:58 [INFO    ] Finished starting DRX
2015-11-10 07:41:58 [INFO    ] Finished starting DRX
2015-11-10 07:41:58 [DEBUG   ] Got command BAM from MCS: ref #535061
2015-11-10 07:41:58 [INFO    ] Finished processing BAM request
2015-11-10 07:41:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535061   8 57336 27718199 A  ERROR'
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 07:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 07:41:58 [DEBUG   ] spawn_process: at 2015-11-10 07:41:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 07:42:06 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 07:42:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:42:10 [DEBUG   ] Finished updating board flags in 0.999 seconds
2015-11-10 07:42:26 [DEBUG   ] Finished updating ant stats in 0.396 seconds
2015-11-10 07:42:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 07:42:49 [DEBUG   ] Got command RPT from MCS: ref #535062
2015-11-10 07:42:49 [DEBUG   ] summary = ERROR
2015-11-10 07:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535062  13 57336 27769059 A  ERRORERROR'
2015-11-10 07:42:49 [DEBUG   ] Got command RPT from MCS: ref #535063
2015-11-10 07:42:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535063  77 57336 27769059 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:43:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 07:43:11 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:43:11 [DEBUG   ] Finished updating board flags in 1.808 seconds
2015-11-10 07:43:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 07:43:47 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2015-11-10 07:43:58 [DEBUG   ] Got command DRX from MCS: ref #535079
2015-11-10 07:43:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:43:58 [DEBUG   ] run_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:43:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535079   8 57336 27838015 A  ERROR'
2015-11-10 07:43:58 [DEBUG   ] Got command DRX from MCS: ref #535080
2015-11-10 07:43:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:43:58 [DEBUG   ] run_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:43:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535080   8 57336 27838019 A  ERROR'
2015-11-10 07:43:58 [INFO    ] Finished starting DRX
2015-11-10 07:43:58 [INFO    ] Finished starting DRX
2015-11-10 07:43:58 [DEBUG   ] Got command BAM from MCS: ref #535081
2015-11-10 07:43:58 [INFO    ] Finished processing BAM request
2015-11-10 07:43:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:43:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535081   8 57336 27838191 A  ERROR'
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 07:43:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 07:43:58 [DEBUG   ] spawn_process: at 2015-11-10 07:43:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 07:44:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:44:10 [DEBUG   ] Finished updating board flags in 1.235 seconds
2015-11-10 07:44:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:44:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:45:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:45:10 [DEBUG   ] Finished updating board flags in 1.241 seconds
2015-11-10 07:45:27 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 07:45:27 [DEBUG   ] Got command RPT from MCS: ref #535086
2015-11-10 07:45:27 [DEBUG   ] summary = ERROR
2015-11-10 07:45:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535086  13 57336 27927635 A  ERRORERROR'
2015-11-10 07:45:27 [DEBUG   ] Got command RPT from MCS: ref #535087
2015-11-10 07:45:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:45:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535087  77 57336 27927839 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:45:28 [DEBUG   ] Got command RPT from MCS: ref #535088
2015-11-10 07:45:28 [DEBUG   ] summary = ERROR
2015-11-10 07:45:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535088  13 57336 27928043 A  ERRORERROR'
2015-11-10 07:45:28 [DEBUG   ] Got command RPT from MCS: ref #535089
2015-11-10 07:45:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:45:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535089  77 57336 27928235 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:45:28 [DEBUG   ] Got command RPT from MCS: ref #535090
2015-11-10 07:45:28 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 07:45:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535090  12 57336 27928439 A  ERRORB�™š'
2015-11-10 07:45:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:45:58 [DEBUG   ] Got command DRX from MCS: ref #535096
2015-11-10 07:45:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:45:58 [DEBUG   ] run_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:45:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535096   8 57336 27958011 A  ERROR'
2015-11-10 07:45:58 [DEBUG   ] Got command DRX from MCS: ref #535097
2015-11-10 07:45:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:45:58 [DEBUG   ] run_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:45:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535097   8 57336 27958015 A  ERROR'
2015-11-10 07:45:58 [INFO    ] Finished starting DRX
2015-11-10 07:45:58 [INFO    ] Finished starting DRX
2015-11-10 07:45:58 [DEBUG   ] Got command BAM from MCS: ref #535098
2015-11-10 07:45:58 [INFO    ] Finished processing BAM request
2015-11-10 07:45:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:45:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535098   8 57336 27958199 A  ERROR'
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 07:45:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 07:45:58 [DEBUG   ] spawn_process: at 2015-11-10 07:45:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 07:46:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:46:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 07:46:10 [DEBUG   ] Finished updating board flags in 1.183 seconds
2015-11-10 07:46:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:46:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:47:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:47:10 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:47:10 [DEBUG   ] Finished updating board flags in 1.039 seconds
2015-11-10 07:47:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:47:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:47:49 [DEBUG   ] Got command RPT from MCS: ref #535102
2015-11-10 07:47:49 [DEBUG   ] summary = ERROR
2015-11-10 07:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535102  13 57336 28069055 A  ERRORERROR'
2015-11-10 07:47:49 [DEBUG   ] Got command RPT from MCS: ref #535103
2015-11-10 07:47:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535103  77 57336 28069055 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:47:58 [DEBUG   ] Got command DRX from MCS: ref #535119
2015-11-10 07:47:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:47:58 [DEBUG   ] run_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:47:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535119   8 57336 28078011 A  ERROR'
2015-11-10 07:47:58 [DEBUG   ] Got command DRX from MCS: ref #535120
2015-11-10 07:47:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:47:58 [DEBUG   ] run_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:47:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535120   8 57336 28078015 A  ERROR'
2015-11-10 07:47:58 [INFO    ] Finished starting DRX
2015-11-10 07:47:58 [INFO    ] Finished starting DRX
2015-11-10 07:47:58 [DEBUG   ] Got command BAM from MCS: ref #535121
2015-11-10 07:47:58 [INFO    ] Finished processing BAM request
2015-11-10 07:47:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535121   8 57336 28078115 A  ERROR'
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 07:47:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 07:47:58 [DEBUG   ] spawn_process: at 2015-11-10 07:47:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 07:48:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:48:11 [DEBUG   ] Finished updating board flags in 1.552 seconds
2015-11-10 07:48:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:48:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:49:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:49:11 [DEBUG   ] Finished updating board flags in 1.433 seconds
2015-11-10 07:49:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:49:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:49:58 [DEBUG   ] Got command DRX from MCS: ref #535122
2015-11-10 07:49:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:49:58 [DEBUG   ] run_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:49:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535122   8 57336 28198007 A  ERROR'
2015-11-10 07:49:58 [INFO    ] Finished starting DRX
2015-11-10 07:49:58 [DEBUG   ] Got command DRX from MCS: ref #535123
2015-11-10 07:49:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:49:58 [DEBUG   ] run_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:49:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535123   8 57336 28198155 A  ERROR'
2015-11-10 07:49:58 [DEBUG   ] Got command BAM from MCS: ref #535124
2015-11-10 07:49:58 [INFO    ] Finished processing BAM request
2015-11-10 07:49:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:49:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535124   8 57336 28198159 A  ERROR'
2015-11-10 07:49:58 [INFO    ] Finished starting DRX
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 07:49:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 07:49:58 [DEBUG   ] spawn_process: at 2015-11-10 07:49:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 07:50:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 07:50:11 [DEBUG   ] Finished updating board flags in 1.328 seconds
2015-11-10 07:50:27 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-10 07:50:27 [DEBUG   ] Got command RPT from MCS: ref #535125
2015-11-10 07:50:27 [DEBUG   ] summary = ERROR
2015-11-10 07:50:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535125  13 57336 28227747 A  ERRORERROR'
2015-11-10 07:50:27 [DEBUG   ] Got command RPT from MCS: ref #535126
2015-11-10 07:50:27 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:50:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535126  77 57336 28227939 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:50:28 [DEBUG   ] Got command RPT from MCS: ref #535127
2015-11-10 07:50:28 [DEBUG   ] summary = ERROR
2015-11-10 07:50:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535127  13 57336 28228143 A  ERRORERROR'
2015-11-10 07:50:28 [DEBUG   ] Got command RPT from MCS: ref #535128
2015-11-10 07:50:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:50:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535128  77 57336 28228347 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:50:28 [DEBUG   ] Got command RPT from MCS: ref #535129
2015-11-10 07:50:28 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 07:50:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535129  12 57336 28228551 A  ERRORB‘™š'
2015-11-10 07:50:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:51:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:51:11 [DEBUG   ] Finished updating board flags in 1.211 seconds
2015-11-10 07:51:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:51:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:51:58 [DEBUG   ] Got command DRX from MCS: ref #535133
2015-11-10 07:51:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:51:58 [DEBUG   ] run_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:51:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535133   8 57336 28318015 A  ERROR'
2015-11-10 07:51:58 [DEBUG   ] Got command DRX from MCS: ref #535134
2015-11-10 07:51:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:51:58 [DEBUG   ] run_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:51:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535134   8 57336 28318019 A  ERROR'
2015-11-10 07:51:58 [INFO    ] Finished starting DRX
2015-11-10 07:51:58 [INFO    ] Finished starting DRX
2015-11-10 07:51:58 [DEBUG   ] Got command BAM from MCS: ref #535135
2015-11-10 07:51:58 [INFO    ] Finished processing BAM request
2015-11-10 07:51:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:51:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535135   8 57336 28318191 A  ERROR'
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 07:51:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 07:51:58 [DEBUG   ] spawn_process: at 2015-11-10 07:51:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 07:52:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:52:11 [DEBUG   ] Finished updating board flags in 1.182 seconds
2015-11-10 07:52:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:52:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:52:49 [DEBUG   ] Got command RPT from MCS: ref #535136
2015-11-10 07:52:49 [DEBUG   ] summary = ERROR
2015-11-10 07:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535136  13 57336 28369055 A  ERRORERROR'
2015-11-10 07:52:49 [DEBUG   ] Got command RPT from MCS: ref #535137
2015-11-10 07:52:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535137  77 57336 28369055 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:53:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:53:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 07:53:11 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:53:11 [DEBUG   ] Finished updating board flags in 1.023 seconds
2015-11-10 07:53:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:53:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:53:58 [DEBUG   ] Got command DRX from MCS: ref #535156
2015-11-10 07:53:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:53:58 [DEBUG   ] run_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:53:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535156   8 57336 28438015 A  ERROR'
2015-11-10 07:53:58 [DEBUG   ] Got command DRX from MCS: ref #535157
2015-11-10 07:53:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:53:58 [DEBUG   ] run_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:53:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535157   8 57336 28438019 A  ERROR'
2015-11-10 07:53:58 [INFO    ] Finished starting DRX
2015-11-10 07:53:58 [INFO    ] Finished starting DRX
2015-11-10 07:53:58 [DEBUG   ] Got command BAM from MCS: ref #535158
2015-11-10 07:53:58 [INFO    ] Finished processing BAM request
2015-11-10 07:53:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535158   8 57336 28438179 A  ERROR'
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 07:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 07:53:58 [DEBUG   ] spawn_process: at 2015-11-10 07:53:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 07:54:08 [DEBUG   ] Finished updating ant stats in 1.801 seconds
2015-11-10 07:54:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:54:11 [DEBUG   ] Finished updating board flags in 1.022 seconds
2015-11-10 07:54:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:54:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:55:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:55:12 [DEBUG   ] Finished updating board flags in 1.982 seconds
2015-11-10 07:55:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:55:27 [DEBUG   ] Got command RPT from MCS: ref #535160
2015-11-10 07:55:27 [DEBUG   ] summary = ERROR
2015-11-10 07:55:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535160  13 57336 28527839 A  ERRORERROR'
2015-11-10 07:55:28 [DEBUG   ] Got command RPT from MCS: ref #535161
2015-11-10 07:55:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:55:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535161  77 57336 28528031 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:55:28 [DEBUG   ] Got command RPT from MCS: ref #535162
2015-11-10 07:55:28 [DEBUG   ] summary = ERROR
2015-11-10 07:55:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535162  13 57336 28528235 A  ERRORERROR'
2015-11-10 07:55:28 [DEBUG   ] Got command RPT from MCS: ref #535163
2015-11-10 07:55:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:55:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535163  77 57336 28528439 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:55:28 [DEBUG   ] Got command RPT from MCS: ref #535164
2015-11-10 07:55:28 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 07:55:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535164  12 57336 28528643 A  ERRORB‘™š'
2015-11-10 07:55:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:55:58 [DEBUG   ] Got command DRX from MCS: ref #535170
2015-11-10 07:55:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:55:58 [DEBUG   ] run_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:55:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535170   8 57336 28558011 A  ERROR'
2015-11-10 07:55:58 [DEBUG   ] Got command DRX from MCS: ref #535171
2015-11-10 07:55:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:55:58 [DEBUG   ] run_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:55:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535171   8 57336 28558015 A  ERROR'
2015-11-10 07:55:58 [INFO    ] Finished starting DRX
2015-11-10 07:55:58 [INFO    ] Finished starting DRX
2015-11-10 07:55:58 [DEBUG   ] Got command BAM from MCS: ref #535172
2015-11-10 07:55:58 [INFO    ] Finished processing BAM request
2015-11-10 07:55:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:55:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535172   8 57336 28558175 A  ERROR'
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 07:55:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 07:55:58 [DEBUG   ] spawn_process: at 2015-11-10 07:55:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 07:56:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:56:11 [DEBUG   ] Finished updating board flags in 1.305 seconds
2015-11-10 07:56:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:56:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:57:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:57:12 [DEBUG   ] Finished updating board flags in 1.765 seconds
2015-11-10 07:57:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:57:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:57:49 [DEBUG   ] Got command RPT from MCS: ref #535176
2015-11-10 07:57:49 [DEBUG   ] summary = ERROR
2015-11-10 07:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535176  13 57336 28669063 A  ERRORERROR'
2015-11-10 07:57:49 [DEBUG   ] Got command RPT from MCS: ref #535177
2015-11-10 07:57:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 07:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535177  77 57336 28669063 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 07:57:58 [DEBUG   ] Got command DRX from MCS: ref #535193
2015-11-10 07:57:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:57:58 [DEBUG   ] run_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:57:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535193   8 57336 28678007 A  ERROR'
2015-11-10 07:57:58 [INFO    ] Finished starting DRX
2015-11-10 07:57:58 [DEBUG   ] Got command DRX from MCS: ref #535194
2015-11-10 07:57:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:57:58 [DEBUG   ] run_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:57:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535194   8 57336 28678155 A  ERROR'
2015-11-10 07:57:58 [DEBUG   ] Got command BAM from MCS: ref #535195
2015-11-10 07:57:58 [INFO    ] Finished processing BAM request
2015-11-10 07:57:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535195   8 57336 28678159 A  ERROR'
2015-11-10 07:57:58 [INFO    ] Finished starting DRX
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 07:57:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 07:57:58 [DEBUG   ] spawn_process: at 2015-11-10 07:57:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 07:58:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:58:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 07:58:11 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 07:58:11 [DEBUG   ] Finished updating board flags in 0.978 seconds
2015-11-10 07:58:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:58:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:59:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:59:12 [DEBUG   ] Finished updating board flags in 1.356 seconds
2015-11-10 07:59:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:59:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 07:59:58 [DEBUG   ] Got command DRX from MCS: ref #535199
2015-11-10 07:59:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:59:58 [DEBUG   ] run_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 07:59:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535199   8 57336 28798007 A  ERROR'
2015-11-10 07:59:58 [INFO    ] Finished starting DRX
2015-11-10 07:59:58 [DEBUG   ] Got command DRX from MCS: ref #535200
2015-11-10 07:59:58 [INFO    ] sending drx command to lwa16
2015-11-10 07:59:58 [DEBUG   ] run_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 07:59:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 07:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535200   8 57336 28798239 A  ERROR'
2015-11-10 07:59:58 [DEBUG   ] Got command BAM from MCS: ref #535201
2015-11-10 07:59:58 [INFO    ] Finished processing BAM request
2015-11-10 07:59:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 07:59:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535201   8 57336 28798243 A  ERROR'
2015-11-10 07:59:58 [INFO    ] Finished starting DRX
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 07:59:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 07:59:58 [DEBUG   ] spawn_process: at 2015-11-10 07:59:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 08:00:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:00:12 [DEBUG   ] Finished updating board flags in 1.222 seconds
2015-11-10 08:00:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:00:27 [DEBUG   ] Got command RPT from MCS: ref #535202
2015-11-10 08:00:27 [DEBUG   ] summary = ERROR
2015-11-10 08:00:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535202  13 57336 28827943 A  ERRORERROR'
2015-11-10 08:00:28 [DEBUG   ] Got command RPT from MCS: ref #535203
2015-11-10 08:00:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:00:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535203  77 57336 28828147 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:00:28 [DEBUG   ] Got command RPT from MCS: ref #535204
2015-11-10 08:00:28 [DEBUG   ] summary = ERROR
2015-11-10 08:00:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535204  13 57336 28828351 A  ERRORERROR'
2015-11-10 08:00:28 [DEBUG   ] Got command RPT from MCS: ref #535205
2015-11-10 08:00:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:00:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535205  77 57336 28828543 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:00:28 [DEBUG   ] Got command RPT from MCS: ref #535206
2015-11-10 08:00:28 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 08:00:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535206  12 57336 28828747 A  ERRORB’™š'
2015-11-10 08:00:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:01:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:01:12 [DEBUG   ] Finished updating board flags in 1.168 seconds
2015-11-10 08:01:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:01:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:01:58 [DEBUG   ] Got command DRX from MCS: ref #535207
2015-11-10 08:01:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:01:58 [DEBUG   ] run_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 08:01:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535207   8 57336 28918015 A  ERROR'
2015-11-10 08:01:58 [DEBUG   ] Got command DRX from MCS: ref #535208
2015-11-10 08:01:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:01:58 [DEBUG   ] run_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 08:01:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535208   8 57336 28918019 A  ERROR'
2015-11-10 08:01:58 [INFO    ] Finished starting DRX
2015-11-10 08:01:58 [INFO    ] Finished starting DRX
2015-11-10 08:01:58 [DEBUG   ] Got command BAM from MCS: ref #535209
2015-11-10 08:01:58 [INFO    ] Finished processing BAM request
2015-11-10 08:01:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 08:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535209   8 57336 28918191 A  ERROR'
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 08:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 08:01:58 [DEBUG   ] spawn_process: at 2015-11-10 08:01:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 08:02:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:02:12 [DEBUG   ] Finished updating board flags in 1.008 seconds
2015-11-10 08:02:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:02:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:02:49 [DEBUG   ] Got command RPT from MCS: ref #535213
2015-11-10 08:02:49 [DEBUG   ] summary = ERROR
2015-11-10 08:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535213  13 57336 28969063 A  ERRORERROR'
2015-11-10 08:02:49 [DEBUG   ] Got command RPT from MCS: ref #535214
2015-11-10 08:02:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535214  77 57336 28969063 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:03:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:03:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 08:03:13 [DEBUG   ] Finished updating board flags in 2.021 seconds
2015-11-10 08:03:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:03:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:03:58 [DEBUG   ] Got command DRX from MCS: ref #535230
2015-11-10 08:03:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:03:58 [DEBUG   ] run_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 08:03:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535230   8 57336 29038011 A  ERROR'
2015-11-10 08:03:58 [DEBUG   ] Got command DRX from MCS: ref #535231
2015-11-10 08:03:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:03:58 [DEBUG   ] run_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 08:03:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535231   8 57336 29038015 A  ERROR'
2015-11-10 08:03:58 [INFO    ] Finished starting DRX
2015-11-10 08:03:58 [INFO    ] Finished starting DRX
2015-11-10 08:03:58 [DEBUG   ] Got command BAM from MCS: ref #535232
2015-11-10 08:03:58 [INFO    ] Finished processing BAM request
2015-11-10 08:03:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 08:03:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535232   8 57336 29038151 A  ERROR'
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2015-11-10 08:03:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2015-11-10 08:03:58 [DEBUG   ] spawn_process: at 2015-11-10 08:03:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2015-11-10 08:04:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:04:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 08:04:12 [DEBUG   ] Finished updating board flags in 1.046 seconds
2015-11-10 08:04:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:04:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:05:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:05:12 [DEBUG   ] Finished updating board flags in 1.485 seconds
2015-11-10 08:05:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:05:28 [DEBUG   ] Got command RPT from MCS: ref #535234
2015-11-10 08:05:28 [DEBUG   ] summary = ERROR
2015-11-10 08:05:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535234  13 57336 29128051 A  ERRORERROR'
2015-11-10 08:05:28 [DEBUG   ] Got command RPT from MCS: ref #535235
2015-11-10 08:05:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:05:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535235  77 57336 29128255 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:05:28 [DEBUG   ] Got command RPT from MCS: ref #535236
2015-11-10 08:05:28 [DEBUG   ] summary = ERROR
2015-11-10 08:05:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535236  13 57336 29128459 A  ERRORERROR'
2015-11-10 08:05:28 [DEBUG   ] Got command RPT from MCS: ref #535237
2015-11-10 08:05:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:05:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535237  77 57336 29128651 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:05:28 [DEBUG   ] Got command RPT from MCS: ref #535238
2015-11-10 08:05:28 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 08:05:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535238  12 57336 29128855 A  ERRORB�™š'
2015-11-10 08:05:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:05:58 [DEBUG   ] Got command DRX from MCS: ref #535247
2015-11-10 08:05:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:05:58 [DEBUG   ] run_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 08:05:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535247   8 57336 29158007 A  ERROR'
2015-11-10 08:05:58 [INFO    ] Finished starting DRX
2015-11-10 08:05:58 [DEBUG   ] Got command DRX from MCS: ref #535248
2015-11-10 08:05:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:05:58 [DEBUG   ] run_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 08:05:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535248   8 57336 29158155 A  ERROR'
2015-11-10 08:05:58 [DEBUG   ] Got command BAM from MCS: ref #535249
2015-11-10 08:05:58 [INFO    ] Finished processing BAM request
2015-11-10 08:05:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 08:05:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535249   8 57336 29158159 A  ERROR'
2015-11-10 08:05:58 [INFO    ] Finished starting DRX
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 08:05:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 08:05:58 [DEBUG   ] spawn_process: at 2015-11-10 08:05:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 08:06:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:06:12 [DEBUG   ] Finished updating board flags in 1.305 seconds
2015-11-10 08:06:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:06:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:07:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:07:13 [DEBUG   ] Finished updating board flags in 1.766 seconds
2015-11-10 08:07:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:07:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:07:49 [DEBUG   ] Got command RPT from MCS: ref #535250
2015-11-10 08:07:49 [DEBUG   ] summary = ERROR
2015-11-10 08:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535250  13 57336 29269059 A  ERRORERROR'
2015-11-10 08:07:49 [DEBUG   ] Got command RPT from MCS: ref #535251
2015-11-10 08:07:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535251  77 57336 29269059 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:07:58 [DEBUG   ] Got command DRX from MCS: ref #535267
2015-11-10 08:07:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:07:58 [DEBUG   ] run_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 08:07:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535267   8 57336 29278015 A  ERROR'
2015-11-10 08:07:58 [DEBUG   ] Got command DRX from MCS: ref #535268
2015-11-10 08:07:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:07:58 [DEBUG   ] run_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 08:07:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535268   8 57336 29278019 A  ERROR'
2015-11-10 08:07:58 [INFO    ] Finished starting DRX
2015-11-10 08:07:58 [INFO    ] Finished starting DRX
2015-11-10 08:07:58 [DEBUG   ] Got command BAM from MCS: ref #535269
2015-11-10 08:07:58 [INFO    ] Finished processing BAM request
2015-11-10 08:07:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 08:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535269   8 57336 29278179 A  ERROR'
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 08:07:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 08:07:58 [DEBUG   ] spawn_process: at 2015-11-10 08:07:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 08:08:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:08:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 08:08:13 [DEBUG   ] Finished updating board flags in 1.587 seconds
2015-11-10 08:08:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:08:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:09:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:09:13 [DEBUG   ] Finished updating board flags in 1.416 seconds
2015-11-10 08:09:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:09:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:09:58 [DEBUG   ] Got command DRX from MCS: ref #535273
2015-11-10 08:09:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:09:58 [DEBUG   ] run_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 08:09:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535273   8 57336 29398011 A  ERROR'
2015-11-10 08:09:58 [DEBUG   ] Got command DRX from MCS: ref #535274
2015-11-10 08:09:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:09:58 [DEBUG   ] run_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 08:09:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535274   8 57336 29398015 A  ERROR'
2015-11-10 08:09:58 [INFO    ] Finished starting DRX
2015-11-10 08:09:58 [INFO    ] Finished starting DRX
2015-11-10 08:09:58 [DEBUG   ] Got command BAM from MCS: ref #535275
2015-11-10 08:09:58 [INFO    ] Finished processing BAM request
2015-11-10 08:09:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 08:09:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535275   8 57336 29398187 A  ERROR'
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 08:09:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 08:09:58 [DEBUG   ] spawn_process: at 2015-11-10 08:09:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 08:10:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:10:13 [DEBUG   ] Finished updating board flags in 1.410 seconds
2015-11-10 08:10:27 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 08:10:28 [DEBUG   ] Got command RPT from MCS: ref #535276
2015-11-10 08:10:28 [DEBUG   ] summary = ERROR
2015-11-10 08:10:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535276  13 57336 29428155 A  ERRORERROR'
2015-11-10 08:10:28 [DEBUG   ] Got command RPT from MCS: ref #535277
2015-11-10 08:10:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:10:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535277  77 57336 29428347 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:10:28 [DEBUG   ] Got command RPT from MCS: ref #535278
2015-11-10 08:10:28 [DEBUG   ] summary = ERROR
2015-11-10 08:10:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535278  13 57336 29428551 A  ERRORERROR'
2015-11-10 08:10:28 [DEBUG   ] Got command RPT from MCS: ref #535279
2015-11-10 08:10:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:10:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535279  77 57336 29428755 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:10:28 [DEBUG   ] Got command RPT from MCS: ref #535280
2015-11-10 08:10:28 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 08:10:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535280  12 57336 29428977 A  ERRORB�™š'
2015-11-10 08:10:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:11:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:11:13 [DEBUG   ] Finished updating board flags in 1.374 seconds
2015-11-10 08:11:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:11:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:11:58 [DEBUG   ] Got command DRX from MCS: ref #535284
2015-11-10 08:11:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:11:58 [DEBUG   ] run_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 08:11:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535284   8 57336 29518011 A  ERROR'
2015-11-10 08:11:58 [DEBUG   ] Got command DRX from MCS: ref #535285
2015-11-10 08:11:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:11:58 [DEBUG   ] run_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 08:11:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535285   8 57336 29518015 A  ERROR'
2015-11-10 08:11:58 [INFO    ] Finished starting DRX
2015-11-10 08:11:58 [INFO    ] Finished starting DRX
2015-11-10 08:11:58 [DEBUG   ] Got command BAM from MCS: ref #535286
2015-11-10 08:11:58 [INFO    ] Finished processing BAM request
2015-11-10 08:11:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 08:11:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535286   8 57336 29518187 A  ERROR'
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 08:11:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 08:11:58 [DEBUG   ] spawn_process: at 2015-11-10 08:11:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 08:12:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:12:13 [DEBUG   ] Finished updating board flags in 1.281 seconds
2015-11-10 08:12:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:12:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:12:49 [DEBUG   ] Got command RPT from MCS: ref #535287
2015-11-10 08:12:49 [DEBUG   ] summary = ERROR
2015-11-10 08:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535287  13 57336 29569067 A  ERRORERROR'
2015-11-10 08:12:49 [DEBUG   ] Got command RPT from MCS: ref #535288
2015-11-10 08:12:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535288  77 57336 29569067 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:13:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:13:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 08:13:13 [DEBUG   ] Finished updating board flags in 1.140 seconds
2015-11-10 08:13:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:13:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:13:58 [DEBUG   ] Got command DRX from MCS: ref #535304
2015-11-10 08:13:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:13:58 [DEBUG   ] run_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 08:13:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535304   8 57336 29638007 A  ERROR'
2015-11-10 08:13:58 [INFO    ] Finished starting DRX
2015-11-10 08:13:58 [DEBUG   ] Got command DRX from MCS: ref #535305
2015-11-10 08:13:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:13:58 [DEBUG   ] run_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 08:13:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535305   8 57336 29638191 A  ERROR'
2015-11-10 08:13:58 [DEBUG   ] Got command BAM from MCS: ref #535306
2015-11-10 08:13:58 [INFO    ] Finished processing BAM request
2015-11-10 08:13:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 08:13:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535306   8 57336 29638195 A  ERROR'
2015-11-10 08:13:58 [INFO    ] Finished starting DRX
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 08:13:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 08:13:58 [DEBUG   ] spawn_process: at 2015-11-10 08:13:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 08:14:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:14:13 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 08:14:13 [DEBUG   ] Finished updating board flags in 1.016 seconds
2015-11-10 08:14:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:14:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:15:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:15:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 08:15:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 08:15:14 [DEBUG   ] Finished updating board flags in 2.009 seconds
2015-11-10 08:15:27 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 08:15:28 [DEBUG   ] Got command RPT from MCS: ref #535311
2015-11-10 08:15:28 [DEBUG   ] summary = ERROR
2015-11-10 08:15:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535311  13 57336 29728275 A  ERRORERROR'
2015-11-10 08:15:28 [DEBUG   ] Got command RPT from MCS: ref #535312
2015-11-10 08:15:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:15:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535312  77 57336 29728479 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:15:28 [DEBUG   ] Got command RPT from MCS: ref #535313
2015-11-10 08:15:28 [DEBUG   ] summary = ERROR
2015-11-10 08:15:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535313  13 57336 29728683 A  ERRORERROR'
2015-11-10 08:15:28 [DEBUG   ] Got command RPT from MCS: ref #535314
2015-11-10 08:15:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:15:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535314  77 57336 29728887 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:15:29 [DEBUG   ] Got command RPT from MCS: ref #535315
2015-11-10 08:15:29 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 08:15:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535315  12 57336 29729079 A  ERRORB‘™š'
2015-11-10 08:15:47 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 08:15:58 [DEBUG   ] Got command DRX from MCS: ref #535321
2015-11-10 08:15:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:15:58 [DEBUG   ] run_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 08:15:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535321   8 57336 29758015 A  ERROR'
2015-11-10 08:15:58 [DEBUG   ] Got command DRX from MCS: ref #535322
2015-11-10 08:15:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:15:58 [DEBUG   ] run_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 08:15:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535322   8 57336 29758019 A  ERROR'
2015-11-10 08:15:58 [INFO    ] Finished starting DRX
2015-11-10 08:15:58 [INFO    ] Finished starting DRX
2015-11-10 08:15:58 [DEBUG   ] Got command BAM from MCS: ref #535323
2015-11-10 08:15:58 [INFO    ] Finished processing BAM request
2015-11-10 08:15:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 08:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535323   8 57336 29758215 A  ERROR'
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2015-11-10 08:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2015-11-10 08:15:58 [DEBUG   ] spawn_process: at 2015-11-10 08:15:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2015-11-10 08:16:07 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 08:16:13 [DEBUG   ] Finished updating board flags in 1.442 seconds
2015-11-10 08:16:27 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 08:16:47 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 08:17:07 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 08:17:13 [DEBUG   ] Finished updating board flags in 1.446 seconds
2015-11-10 08:17:27 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 08:17:47 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 08:17:49 [DEBUG   ] Got command RPT from MCS: ref #535327
2015-11-10 08:17:49 [DEBUG   ] summary = ERROR
2015-11-10 08:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535327  13 57336 29869071 A  ERRORERROR'
2015-11-10 08:17:49 [DEBUG   ] Got command RPT from MCS: ref #535328
2015-11-10 08:17:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535328  77 57336 29869071 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:17:58 [DEBUG   ] Got command DRX from MCS: ref #535344
2015-11-10 08:17:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:17:58 [DEBUG   ] run_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 36000000.0 7 9 0']
2015-11-10 08:17:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535344   8 57336 29878015 A  ERROR'
2015-11-10 08:17:58 [DEBUG   ] Got command DRX from MCS: ref #535345
2015-11-10 08:17:58 [INFO    ] sending drx command to lwa16
2015-11-10 08:17:58 [DEBUG   ] run_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 54000000.0 7 9 0']
2015-11-10 08:17:58 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535345   8 57336 29878019 A  ERROR'
2015-11-10 08:17:58 [INFO    ] Finished starting DRX
2015-11-10 08:17:58 [INFO    ] Finished starting DRX
2015-11-10 08:17:58 [DEBUG   ] Got command BAM from MCS: ref #535346
2015-11-10 08:17:58 [INFO    ] Finished processing BAM request
2015-11-10 08:17:58 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 08:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535346   8 57336 29878191 A  ERROR'
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 08:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 08:17:58 [DEBUG   ] spawn_process: at 2015-11-10 08:17:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 08:18:07 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 08:18:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 08:18:14 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 08:18:14 [DEBUG   ] Finished updating board flags in 1.943 seconds
2015-11-10 08:18:28 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-10 08:18:47 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 08:19:07 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 08:19:13 [DEBUG   ] Finished updating board flags in 1.232 seconds
2015-11-10 08:19:27 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 08:19:47 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 08:20:00 [DEBUG   ] Got command STP from MCS: ref #535347
2015-11-10 08:20:00 [INFO    ] Finished processing BAM request
2015-11-10 08:20:00 [INFO    ] Shutdown the adder chain on beam #4
2015-11-10 08:20:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   535347   8 57336 30000027 A  ERROR'
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2015-11-10 08:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2015-11-10 08:20:00 [DEBUG   ] spawn_process: at 2015-11-10 08:20:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2015-11-10 08:20:07 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 08:20:14 [DEBUG   ] Finished updating board flags in 1.761 seconds
2015-11-10 08:20:27 [DEBUG   ] Finished updating ant stats in 0.397 seconds
2015-11-10 08:20:28 [DEBUG   ] Got command RPT from MCS: ref #535348
2015-11-10 08:20:28 [DEBUG   ] summary = ERROR
2015-11-10 08:20:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535348  13 57336 30028383 A  ERRORERROR'
2015-11-10 08:20:28 [DEBUG   ] Got command RPT from MCS: ref #535349
2015-11-10 08:20:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:20:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535349  77 57336 30028587 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:20:28 [DEBUG   ] Got command RPT from MCS: ref #535350
2015-11-10 08:20:28 [DEBUG   ] summary = ERROR
2015-11-10 08:20:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535350  13 57336 30028779 A  ERRORERROR'
2015-11-10 08:20:28 [DEBUG   ] Got command RPT from MCS: ref #535351
2015-11-10 08:20:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:20:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535351  77 57336 30028983 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:20:29 [DEBUG   ] Got command RPT from MCS: ref #535352
2015-11-10 08:20:29 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 08:20:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535352  12 57336 30029187 A  ERRORB�™š'
2015-11-10 08:20:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 08:21:08 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 08:21:13 [DEBUG   ] Finished updating board flags in 1.010 seconds
2015-11-10 08:21:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 08:21:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 08:22:08 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 08:22:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 08:22:13 [DEBUG   ] Finished updating board flags in 1.027 seconds
2015-11-10 08:22:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 08:22:48 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 08:22:49 [DEBUG   ] Got command RPT from MCS: ref #535356
2015-11-10 08:22:49 [DEBUG   ] summary = ERROR
2015-11-10 08:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535356  13 57336 30169071 A  ERRORERROR'
2015-11-10 08:22:49 [DEBUG   ] Got command RPT from MCS: ref #535357
2015-11-10 08:22:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535357  77 57336 30169071 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:23:08 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 08:23:13 [DEBUG   ] Finished updating board flags in 0.981 seconds
2015-11-10 08:23:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 08:23:48 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 08:24:08 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 08:24:14 [DEBUG   ] Finished updating board flags in 1.356 seconds
2015-11-10 08:24:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 08:24:48 [DEBUG   ] Finished updating ant stats in 1.193 seconds
2015-11-10 08:25:08 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 08:25:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 08:25:14 [DEBUG   ] Finished updating board flags in 1.200 seconds
2015-11-10 08:25:28 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 08:25:28 [DEBUG   ] Got command RPT from MCS: ref #535377
2015-11-10 08:25:28 [DEBUG   ] summary = ERROR
2015-11-10 08:25:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535377  13 57336 30328475 A  ERRORERROR'
2015-11-10 08:25:28 [DEBUG   ] Got command RPT from MCS: ref #535378
2015-11-10 08:25:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:25:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535378  77 57336 30328679 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:25:28 [DEBUG   ] Got command RPT from MCS: ref #535379
2015-11-10 08:25:28 [DEBUG   ] summary = ERROR
2015-11-10 08:25:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535379  13 57336 30328871 A  ERRORERROR'
2015-11-10 08:25:29 [DEBUG   ] Got command RPT from MCS: ref #535380
2015-11-10 08:25:29 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:25:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535380  77 57336 30329075 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:25:29 [DEBUG   ] Got command RPT from MCS: ref #535381
2015-11-10 08:25:29 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 08:25:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535381  12 57336 30329279 A  ERRORBŒÌÍ'
2015-11-10 08:25:48 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 08:26:08 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 08:26:14 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 08:26:14 [DEBUG   ] Finished updating board flags in 1.058 seconds
2015-11-10 08:26:28 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 08:26:48 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 08:27:08 [DEBUG   ] Finished updating ant stats in 0.993 seconds
2015-11-10 08:27:15 [DEBUG   ] Finished updating board flags in 1.995 seconds
2015-11-10 08:27:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:27:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:27:49 [DEBUG   ] Got command RPT from MCS: ref #535390
2015-11-10 08:27:49 [DEBUG   ] summary = ERROR
2015-11-10 08:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535390  13 57336 30469071 A  ERRORERROR'
2015-11-10 08:27:49 [DEBUG   ] Got command RPT from MCS: ref #535391
2015-11-10 08:27:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535391  77 57336 30469071 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:28:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 08:28:14 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 08:28:14 [DEBUG   ] Finished updating board flags in 1.192 seconds
2015-11-10 08:28:28 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 08:28:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:29:08 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:29:14 [DEBUG   ] Finished updating board flags in 1.028 seconds
2015-11-10 08:29:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:29:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:30:08 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:30:15 [DEBUG   ] Finished updating board flags in 1.550 seconds
2015-11-10 08:30:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:30:28 [DEBUG   ] Got command RPT from MCS: ref #535410
2015-11-10 08:30:28 [DEBUG   ] summary = ERROR
2015-11-10 08:30:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535410  13 57336 30628551 A  ERRORERROR'
2015-11-10 08:30:28 [DEBUG   ] Got command RPT from MCS: ref #535411
2015-11-10 08:30:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:30:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535411  77 57336 30628755 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:30:28 [DEBUG   ] Got command RPT from MCS: ref #535412
2015-11-10 08:30:28 [DEBUG   ] summary = ERROR
2015-11-10 08:30:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535412  13 57336 30628973 A  ERRORERROR'
2015-11-10 08:30:29 [DEBUG   ] Got command RPT from MCS: ref #535413
2015-11-10 08:30:29 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:30:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535413  77 57336 30629163 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:30:29 [DEBUG   ] Got command RPT from MCS: ref #535414
2015-11-10 08:30:29 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 08:30:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535414  12 57336 30629355 A  ERRORBŒÌÍ'
2015-11-10 08:30:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:31:08 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:31:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 08:31:15 [DEBUG   ] Finished updating board flags in 1.372 seconds
2015-11-10 08:31:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:31:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:32:08 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:32:15 [DEBUG   ] Finished updating board flags in 1.219 seconds
2015-11-10 08:32:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:32:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:32:49 [DEBUG   ] Got command RPT from MCS: ref #535418
2015-11-10 08:32:49 [DEBUG   ] summary = ERROR
2015-11-10 08:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535418  13 57336 30769071 A  ERRORERROR'
2015-11-10 08:32:49 [DEBUG   ] Got command RPT from MCS: ref #535419
2015-11-10 08:32:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535419  77 57336 30769071 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:33:08 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:33:15 [DEBUG   ] Finished updating board flags in 1.213 seconds
2015-11-10 08:33:28 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 08:33:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 08:34:08 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 08:34:15 [DEBUG   ] Finished updating board flags in 1.234 seconds
2015-11-10 08:34:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:34:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:35:08 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 08:35:15 [DEBUG   ] Finished updating board flags in 1.236 seconds
2015-11-10 08:35:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:35:28 [DEBUG   ] Got command RPT from MCS: ref #535436
2015-11-10 08:35:28 [DEBUG   ] summary = ERROR
2015-11-10 08:35:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535436  13 57336 30928635 A  ERRORERROR'
2015-11-10 08:35:28 [DEBUG   ] Got command RPT from MCS: ref #535437
2015-11-10 08:35:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:35:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535437  77 57336 30928839 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:35:29 [DEBUG   ] Got command RPT from MCS: ref #535438
2015-11-10 08:35:29 [DEBUG   ] summary = ERROR
2015-11-10 08:35:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535438  13 57336 30929043 A  ERRORERROR'
2015-11-10 08:35:29 [DEBUG   ] Got command RPT from MCS: ref #535439
2015-11-10 08:35:29 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:35:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535439  77 57336 30929247 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:35:29 [DEBUG   ] Got command RPT from MCS: ref #535440
2015-11-10 08:35:29 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 08:35:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535440  12 57336 30929439 A  ERRORBŒÌÍ'
2015-11-10 08:35:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 08:36:08 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:36:15 [DEBUG   ] Finished updating board flags in 1.120 seconds
2015-11-10 08:36:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:36:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 08:37:08 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:37:15 [DEBUG   ] Finished updating board flags in 1.024 seconds
2015-11-10 08:37:28 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 08:37:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 08:37:49 [DEBUG   ] Got command RPT from MCS: ref #535449
2015-11-10 08:37:49 [DEBUG   ] summary = ERROR
2015-11-10 08:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535449  13 57336 31069067 A  ERRORERROR'
2015-11-10 08:37:49 [DEBUG   ] Got command RPT from MCS: ref #535450
2015-11-10 08:37:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535450  77 57336 31069067 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:38:08 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:38:16 [DEBUG   ] Finished updating board flags in 1.997 seconds
2015-11-10 08:38:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:38:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 08:39:08 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 08:39:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 08:39:16 [DEBUG   ] Finished updating board flags in 1.785 seconds
2015-11-10 08:39:28 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 08:39:48 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 08:40:08 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2015-11-10 08:40:15 [DEBUG   ] Finished updating board flags in 1.083 seconds
2015-11-10 08:40:28 [DEBUG   ] Finished updating ant stats in 0.365 seconds
2015-11-10 08:40:28 [DEBUG   ] Got command RPT from MCS: ref #535469
2015-11-10 08:40:28 [DEBUG   ] summary = ERROR
2015-11-10 08:40:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535469  13 57336 31228727 A  ERRORERROR'
2015-11-10 08:40:28 [DEBUG   ] Got command RPT from MCS: ref #535470
2015-11-10 08:40:28 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:40:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535470  77 57336 31228931 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:40:29 [DEBUG   ] Got command RPT from MCS: ref #535471
2015-11-10 08:40:29 [DEBUG   ] summary = ERROR
2015-11-10 08:40:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535471  13 57336 31229135 A  ERRORERROR'
2015-11-10 08:40:29 [DEBUG   ] Got command RPT from MCS: ref #535472
2015-11-10 08:40:29 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:40:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535472  77 57336 31229327 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:40:29 [DEBUG   ] Got command RPT from MCS: ref #535473
2015-11-10 08:40:29 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2015-11-10 08:40:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535473  12 57336 31229531 A  ERRORB‹ÌÍ'
2015-11-10 08:40:49 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 08:41:09 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 08:41:15 [DEBUG   ] Finished updating board flags in 0.993 seconds
2015-11-10 08:41:29 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 08:41:49 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 08:42:09 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 08:42:16 [DEBUG   ] Finished updating board flags in 1.389 seconds
2015-11-10 08:42:29 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 08:42:49 [DEBUG   ] Got command RPT from MCS: ref #535477
2015-11-10 08:42:49 [DEBUG   ] summary = ERROR
2015-11-10 08:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535477  13 57336 31369075 A  ERRORERROR'
2015-11-10 08:42:49 [DEBUG   ] Got command RPT from MCS: ref #535478
2015-11-10 08:42:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2015-11-10 08:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535478  77 57336 31369075 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2015-11-10 08:42:49 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 08:43:09 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 08:43:16 [DEBUG   ] Finished updating board flags in 1.129 seconds
2015-11-10 08:43:29 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 08:43:49 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 08:44:00 [DEBUG   ] Got command INI from MCS: ref #535494
2015-11-10 08:44:00 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2015-11-10 08:44:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   535494   8 57336 31440291 ABOOTING'
2015-11-10 08:44:20 [INFO    ] -------------------
2015-11-10 08:44:20 [INFO    ]  LWA board status
2015-11-10 08:44:20 [INFO    ] -------------------
2015-11-10 08:44:20 [INFO    ] DP 1: 2 lwa27
2015-11-10 08:44:20 [INFO    ] DP 1: 3 lwa24
2015-11-10 08:44:20 [INFO    ] DP 1: 4 lwa23
2015-11-10 08:44:20 [INFO    ] DP 1: 5 lwa18
2015-11-10 08:44:20 [INFO    ] DP 1: 6 lwa10
2015-11-10 08:44:20 [INFO    ] DP 1: 7 lwa32
2015-11-10 08:44:20 [INFO    ] DP 1: 8 lwa11
2015-11-10 08:44:20 [INFO    ] DP 1: 9 lwa30
2015-11-10 08:44:20 [INFO    ] DP 1: 10 lwa12
2015-11-10 08:44:20 [INFO    ] DP 1: 11 lwa8
2015-11-10 08:44:20 [INFO    ] DP 1: 12 lwa13
2015-11-10 08:44:20 [INFO    ] DP 1: 13 lwa26
2015-11-10 08:44:20 [INFO    ] DP 1: 14 lwa14
2015-11-10 08:44:20 [INFO    ] DP 1: 16 lwa22
2015-11-10 08:44:20 [INFO    ] DP 1: 17 lwa20
2015-11-10 08:44:20 [INFO    ] DP 1: 18 lwa1
2015-11-10 08:44:20 [INFO    ] DP 1: 19 lwa5
2015-11-10 08:44:20 [INFO    ] DP 1: 20 lwa25
2015-11-10 08:44:20 [INFO    ] DP 1: 21 lwa35
2015-11-10 08:44:20 [INFO    ] DP 1: 22 lwa19
2015-11-10 08:44:20 [INFO    ] DP 1: 23 lwa21
2015-11-10 08:44:20 [INFO    ] DP 1: 24 lwa6
2015-11-10 08:44:20 [INFO    ] DP 1: 25 lwa7
2015-11-10 08:44:20 [INFO    ] DP 1: 26 lwa33
2015-11-10 08:44:20 [INFO    ] DP 1: 27 lwa15
2015-11-10 08:44:20 [INFO    ] DP 1: 28 lwa28
2015-11-10 08:44:20 [INFO    ] DP 2: 1 lwa16
2015-11-10 08:44:20 [INFO    ] DP 2: 15 lwa34
2015-11-10 08:44:20 [INFO    ] Total of 28 boards
2015-11-10 08:44:20 [INFO    ] Total of 260 stands
2015-11-10 08:44:20 [INFO    ] -------------------
2015-11-10 08:44:20 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19095
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19096
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19097
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19098
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19099
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19100
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19101
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19102
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19103
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19104
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19105
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19106
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19107
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19108
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19109
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19110
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19111
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19112
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19113
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19114
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19115
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19116
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19117
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19118
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19119
2015-11-10 08:44:27 [DEBUG   ] 2015-11-10 08:44:27 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19120
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19095 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19096 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19097 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19098 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19099 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19100 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19101 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19102 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19103 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19104 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19105 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19106 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19107 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19108 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19109 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19110 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19111 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19112 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19113 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19114 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19115 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19116 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19117 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19118 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19119 exited with status 0
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19120 exited with status 0
2015-11-10 08:44:39 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] spawn_process: at 2015-11-10 08:44:39 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 19147
2015-11-10 08:44:39 [DEBUG   ] 2015-11-10 08:44:39 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 19148
2015-11-10 08:44:45 [DEBUG   ] 2015-11-10 08:44:45 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 19147 exited with status 0
2015-11-10 08:44:45 [DEBUG   ] 2015-11-10 08:44:45 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 19148 exited with status 0
2015-11-10 08:44:45 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2015-11-10 08:44:45 [DEBUG   ] spawn_process: at 2015-11-10 08:44:45 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2015-11-10 08:44:45 [DEBUG   ] spawn_process: at 2015-11-10 08:44:45 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2015-11-10 08:44:45 [INFO    ] Setting Beam 1 xbars
2015-11-10 08:44:45 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2015-11-10 08:44:45 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2015-11-10 08:44:45 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2015-11-10 08:44:45 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2015-11-10 08:44:45 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2015-11-10 08:44:46 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2015-11-10 08:44:46 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2015-11-10 08:44:46 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2015-11-10 08:44:46 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2015-11-10 08:44:46 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2015-11-10 08:44:46 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2015-11-10 08:44:46 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2015-11-10 08:44:46 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2015-11-10 08:44:46 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2015-11-10 08:44:47 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2015-11-10 08:44:47 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2015-11-10 08:44:47 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2015-11-10 08:44:47 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2015-11-10 08:44:47 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2015-11-10 08:44:47 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2015-11-10 08:44:47 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2015-11-10 08:44:47 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2015-11-10 08:44:47 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2015-11-10 08:44:48 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2015-11-10 08:44:48 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2015-11-10 08:44:48 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2015-11-10 08:44:48 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2015-11-10 08:44:48 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2015-11-10 08:44:48 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2015-11-10 08:44:48 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2015-11-10 08:44:48 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2015-11-10 08:44:48 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2015-11-10 08:44:49 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2015-11-10 08:44:49 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2015-11-10 08:44:49 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2015-11-10 08:44:49 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2015-11-10 08:44:49 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2015-11-10 08:44:49 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2015-11-10 08:44:49 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2015-11-10 08:44:49 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2015-11-10 08:44:49 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2015-11-10 08:44:50 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2015-11-10 08:44:50 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2015-11-10 08:44:50 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2015-11-10 08:44:50 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2015-11-10 08:44:50 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2015-11-10 08:44:50 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2015-11-10 08:44:50 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2015-11-10 08:44:50 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2015-11-10 08:44:50 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2015-11-10 08:44:50 [INFO    ] Setting Beam 2 xbars
2015-11-10 08:44:50 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2015-11-10 08:44:51 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2015-11-10 08:44:51 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2015-11-10 08:44:51 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2015-11-10 08:44:51 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2015-11-10 08:44:51 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2015-11-10 08:44:51 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2015-11-10 08:44:51 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2015-11-10 08:44:51 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2015-11-10 08:44:51 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2015-11-10 08:44:52 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2015-11-10 08:44:52 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2015-11-10 08:44:52 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2015-11-10 08:44:52 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2015-11-10 08:44:52 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2015-11-10 08:44:52 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2015-11-10 08:44:52 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2015-11-10 08:44:52 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2015-11-10 08:44:52 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2015-11-10 08:44:53 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2015-11-10 08:44:53 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2015-11-10 08:44:53 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2015-11-10 08:44:53 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2015-11-10 08:44:53 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2015-11-10 08:44:53 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2015-11-10 08:44:53 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2015-11-10 08:44:53 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2015-11-10 08:44:53 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2015-11-10 08:44:54 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2015-11-10 08:44:54 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2015-11-10 08:44:54 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2015-11-10 08:44:54 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2015-11-10 08:44:54 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2015-11-10 08:44:54 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2015-11-10 08:44:54 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2015-11-10 08:44:54 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2015-11-10 08:44:54 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2015-11-10 08:44:55 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2015-11-10 08:44:55 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2015-11-10 08:44:55 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2015-11-10 08:44:55 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2015-11-10 08:44:55 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2015-11-10 08:44:55 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2015-11-10 08:44:55 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2015-11-10 08:44:55 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2015-11-10 08:44:55 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2015-11-10 08:44:56 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2015-11-10 08:44:56 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2015-11-10 08:44:56 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2015-11-10 08:44:56 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2015-11-10 08:44:56 [INFO    ] Setting Beam 3 xbars
2015-11-10 08:44:56 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2015-11-10 08:44:56 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2015-11-10 08:44:56 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2015-11-10 08:44:56 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2015-11-10 08:44:56 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2015-11-10 08:44:56 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2015-11-10 08:44:57 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2015-11-10 08:44:57 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2015-11-10 08:44:57 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2015-11-10 08:44:57 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2015-11-10 08:44:57 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2015-11-10 08:44:57 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2015-11-10 08:44:57 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2015-11-10 08:44:57 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2015-11-10 08:44:57 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2015-11-10 08:44:58 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2015-11-10 08:44:58 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2015-11-10 08:44:58 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2015-11-10 08:44:58 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2015-11-10 08:44:58 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2015-11-10 08:44:58 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2015-11-10 08:44:58 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2015-11-10 08:44:58 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2015-11-10 08:44:58 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2015-11-10 08:44:59 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2015-11-10 08:44:59 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2015-11-10 08:44:59 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2015-11-10 08:44:59 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2015-11-10 08:44:59 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2015-11-10 08:44:59 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2015-11-10 08:44:59 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2015-11-10 08:44:59 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2015-11-10 08:44:59 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2015-11-10 08:45:00 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2015-11-10 08:45:00 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2015-11-10 08:45:00 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2015-11-10 08:45:00 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2015-11-10 08:45:00 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2015-11-10 08:45:00 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2015-11-10 08:45:00 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2015-11-10 08:45:00 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2015-11-10 08:45:00 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2015-11-10 08:45:01 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2015-11-10 08:45:01 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2015-11-10 08:45:01 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2015-11-10 08:45:01 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2015-11-10 08:45:01 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2015-11-10 08:45:01 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2015-11-10 08:45:01 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2015-11-10 08:45:01 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2015-11-10 08:45:01 [INFO    ] Setting Beam 4 xbars
2015-11-10 08:45:01 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2015-11-10 08:45:02 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2015-11-10 08:45:02 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2015-11-10 08:45:02 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2015-11-10 08:45:02 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2015-11-10 08:45:02 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2015-11-10 08:45:02 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2015-11-10 08:45:02 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2015-11-10 08:45:02 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2015-11-10 08:45:02 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2015-11-10 08:45:03 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2015-11-10 08:45:03 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2015-11-10 08:45:03 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2015-11-10 08:45:03 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2015-11-10 08:45:03 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2015-11-10 08:45:03 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2015-11-10 08:45:03 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2015-11-10 08:45:03 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2015-11-10 08:45:03 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2015-11-10 08:45:04 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2015-11-10 08:45:04 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2015-11-10 08:45:04 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2015-11-10 08:45:04 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2015-11-10 08:45:04 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2015-11-10 08:45:04 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2015-11-10 08:45:04 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2015-11-10 08:45:04 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2015-11-10 08:45:04 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2015-11-10 08:45:05 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2015-11-10 08:45:05 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2015-11-10 08:45:05 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2015-11-10 08:45:05 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2015-11-10 08:45:05 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2015-11-10 08:45:05 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2015-11-10 08:45:05 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2015-11-10 08:45:05 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2015-11-10 08:45:05 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2015-11-10 08:45:06 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2015-11-10 08:45:06 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2015-11-10 08:45:06 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2015-11-10 08:45:06 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2015-11-10 08:45:06 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2015-11-10 08:45:06 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2015-11-10 08:45:06 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2015-11-10 08:45:06 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2015-11-10 08:45:06 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2015-11-10 08:45:07 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2015-11-10 08:45:07 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2015-11-10 08:45:07 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2015-11-10 08:45:07 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2015-11-10 08:45:07 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2015-11-10 08:45:07 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2015-11-10 08:45:07 [INFO    ] All xbars set
2015-11-10 08:45:07 [INFO    ] Setting adder chain routing for all beams
2015-11-10 08:45:12 [INFO    ] Setting beam ids
2015-11-10 08:45:22 [INFO    ] Starting beam delay calibration
2015-11-10 08:45:28 [DEBUG   ] Got command RPT from MCS: ref #535537
2015-11-10 08:45:28 [DEBUG   ] summary = BOOTING
2015-11-10 08:45:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535537  15 57336 31528811 ABOOTINGBOOTING'
2015-11-10 08:45:34 [INFO    ] Beam 1 latency is 5759
2015-11-10 08:45:34 [INFO    ] Beam 2 latency is 5753
2015-11-10 08:45:34 [INFO    ] Beam 3 latency is 5743
2015-11-10 08:45:34 [INFO    ] Beam 4 latency is 5738
2015-11-10 08:45:34 [INFO    ] Stop calibration
2015-11-10 08:45:42 [INFO    ] Beam delay calibration successful
2015-11-10 08:45:42 [INFO    ] Calibration passed
2015-11-10 08:45:42 [DEBUG   ] spawn_process: at 2015-11-10 08:45:42 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2015-11-10 08:45:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2015-11-10 08:45:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2015-11-10 08:45:42 [DEBUG   ] spawn_process: at 2015-11-10 08:45:42 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2015-11-10 08:45:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2015-11-10 08:45:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2015-11-10 08:45:42 [DEBUG   ] spawn_process: at 2015-11-10 08:45:42 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2015-11-10 08:45:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2015-11-10 08:45:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2015-11-10 08:45:42 [DEBUG   ] spawn_process: at 2015-11-10 08:45:42 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2015-11-10 08:45:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2015-11-10 08:45:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2015-11-10 08:45:42 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -3 msec
2015-11-10 08:45:43 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 283 ms (1447145143.285 s) with PID 20728 and time 1447145144
2015-11-10 08:45:43 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2015-11-10 08:45:43 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 371 ms (1447145143.373 s) with PID 20731 and time 1447145144
2015-11-10 08:45:44 [DEBUG   ] Finished updating ant stats in 0.873 seconds
2015-11-10 08:45:44 [INFO    ] Started antennas statistics background thread
2015-11-10 08:45:44 [INFO    ] Started board flags background thread
2015-11-10 08:45:44 [INFO    ] Started BAM background thread
2015-11-10 08:45:44 [INFO    ] Finished the INI process (ref. #535494) in 104.206 s
2015-11-10 08:45:45 [DEBUG   ] Finished updating board flags in 1.070 seconds
2015-11-10 08:46:04 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 08:46:24 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 08:46:30 [DEBUG   ] Got command RPT from MCS: ref #535543
2015-11-10 08:46:30 [DEBUG   ] summary = NORMAL
2015-11-10 08:46:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535543  14 57336 31590287 A NORMALNORMAL'
2015-11-10 08:46:30 [DEBUG   ] Got command RPT from MCS: ref #535544
2015-11-10 08:46:30 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 08:46:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535544  48 57336 31590287 A NORMALSystem calibrated and operating normally'
2015-11-10 08:46:44 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 08:46:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 08:46:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-10 08:46:45 [DEBUG   ] Finished updating board flags in 0.994 seconds
2015-11-10 08:47:04 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 08:47:24 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 08:47:44 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 08:47:45 [DEBUG   ] Finished updating board flags in 1.276 seconds
2015-11-10 08:47:49 [DEBUG   ] Got command RPT from MCS: ref #535548
2015-11-10 08:47:49 [DEBUG   ] summary = NORMAL
2015-11-10 08:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535548  14 57336 31669067 A NORMALNORMAL'
2015-11-10 08:47:49 [DEBUG   ] Got command RPT from MCS: ref #535549
2015-11-10 08:47:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 08:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535549  48 57336 31669067 A NORMALSystem calibrated and operating normally'
2015-11-10 08:48:04 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 08:48:24 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 08:48:44 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 08:48:45 [DEBUG   ] Finished updating board flags in 1.248 seconds
2015-11-10 08:49:04 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 08:49:24 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 08:49:44 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 08:49:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 08:49:46 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 08:49:46 [DEBUG   ] Finished updating board flags in 1.766 seconds
2015-11-10 08:50:04 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 08:50:24 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 08:50:28 [DEBUG   ] Got command RPT from MCS: ref #535565
2015-11-10 08:50:28 [DEBUG   ] summary = NORMAL
2015-11-10 08:50:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535565  14 57336 31828931 A NORMALNORMAL'
2015-11-10 08:50:44 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 08:50:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2015-11-10 08:50:45 [DEBUG   ] Finished updating board flags in 0.967 seconds
2015-11-10 08:51:04 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 08:51:24 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 08:51:44 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2015-11-10 08:51:46 [DEBUG   ] Finished updating board flags in 1.358 seconds
2015-11-10 08:52:04 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2015-11-10 08:52:24 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2015-11-10 08:52:44 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2015-11-10 08:52:46 [DEBUG   ] Finished updating board flags in 1.190 seconds
2015-11-10 08:52:49 [DEBUG   ] Got command RPT from MCS: ref #535569
2015-11-10 08:52:49 [DEBUG   ] summary = NORMAL
2015-11-10 08:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535569  14 57336 31969071 A NORMALNORMAL'
2015-11-10 08:52:49 [DEBUG   ] Got command RPT from MCS: ref #535570
2015-11-10 08:52:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 08:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535570  48 57336 31969071 A NORMALSystem calibrated and operating normally'
2015-11-10 08:53:04 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2015-11-10 08:53:24 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2015-11-10 08:53:44 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 08:53:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 08:53:46 [DEBUG   ] Finished updating board flags in 1.021 seconds
2015-11-10 08:54:05 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 08:54:25 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 08:54:45 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 08:54:47 [DEBUG   ] Finished updating board flags in 2.000 seconds
2015-11-10 08:55:05 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 08:55:25 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 08:55:29 [DEBUG   ] Got command RPT from MCS: ref #535590
2015-11-10 08:55:29 [DEBUG   ] summary = NORMAL
2015-11-10 08:55:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535590  14 57336 32129051 A NORMALNORMAL'
2015-11-10 08:55:45 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 08:55:46 [DEBUG   ] Finished updating board flags in 1.235 seconds
2015-11-10 08:56:05 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 08:56:25 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 08:56:45 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 08:56:46 [DEBUG   ] Finished updating board flags in 1.227 seconds
2015-11-10 08:56:59 [DEBUG   ] Got command DRX from MCS: ref #535607
2015-11-10 08:56:59 [INFO    ] sending drx command to lwa16
2015-11-10 08:56:59 [DEBUG   ] run_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 32000000.0 7 9 10']
2015-11-10 08:56:59 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:56:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535607   8 57336 32219015 A NORMAL'
2015-11-10 08:56:59 [DEBUG   ] Got command DRX from MCS: ref #535605
2015-11-10 08:56:59 [INFO    ] sending drx command to lwa34
2015-11-10 08:56:59 [DEBUG   ] run_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa34', '/lwa/software/drx 1 1 32000000.0 7 9 10']
2015-11-10 08:56:59 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:56:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535605   8 57336 32219019 A NORMAL'
2015-11-10 08:56:59 [DEBUG   ] Got command DRX from MCS: ref #535606
2015-11-10 08:56:59 [INFO    ] sending drx command to lwa16
2015-11-10 08:56:59 [DEBUG   ] run_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 32000000.0 7 9 10']
2015-11-10 08:56:59 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:56:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535606   8 57336 32219023 A NORMAL'
2015-11-10 08:56:59 [DEBUG   ] Got command DRX from MCS: ref #535608
2015-11-10 08:56:59 [INFO    ] sending drx command to lwa34
2015-11-10 08:56:59 [DEBUG   ] run_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa34', '/lwa/software/drx 1 2 46000000.0 7 9 10']
2015-11-10 08:56:59 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:56:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535608   8 57336 32219027 A NORMAL'
2015-11-10 08:56:59 [DEBUG   ] Got command DRX from MCS: ref #535609
2015-11-10 08:56:59 [INFO    ] sending drx command to lwa16
2015-11-10 08:56:59 [DEBUG   ] run_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 46000000.0 7 9 10']
2015-11-10 08:56:59 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:56:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535609   8 57336 32219031 A NORMAL'
2015-11-10 08:56:59 [DEBUG   ] Got command DRX from MCS: ref #535610
2015-11-10 08:56:59 [INFO    ] sending drx command to lwa16
2015-11-10 08:56:59 [DEBUG   ] run_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 46000000.0 7 9 10']
2015-11-10 08:56:59 [DEBUG   ] drx = exited with status True, code 0x00
2015-11-10 08:56:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   535610   8 57336 32219035 A NORMAL'
2015-11-10 08:56:59 [INFO    ] Finished starting DRX
2015-11-10 08:56:59 [INFO    ] Finished starting DRX
2015-11-10 08:56:59 [INFO    ] Finished starting DRX
2015-11-10 08:56:59 [INFO    ] Finished starting DRX
2015-11-10 08:56:59 [INFO    ] Finished starting DRX
2015-11-10 08:56:59 [INFO    ] Finished starting DRX
2015-11-10 08:56:59 [DEBUG   ] Got command BAM from MCS: ref #535613
2015-11-10 08:56:59 [INFO    ] Finished processing BAM request
2015-11-10 08:56:59 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 08:56:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535613   8 57336 32219195 A NORMAL'
2015-11-10 08:56:59 [DEBUG   ] Got command BAM from MCS: ref #535611
2015-11-10 08:56:59 [INFO    ] Finished processing BAM request
2015-11-10 08:56:59 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 08:56:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535611   8 57336 32219343 A NORMAL'
2015-11-10 08:56:59 [DEBUG   ] Got command BAM from MCS: ref #535612
2015-11-10 08:56:59 [INFO    ] Finished processing BAM request
2015-11-10 08:56:59 [DEBUG   ] bam = exited with status True, code 0x00
2015-11-10 08:56:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   535612   8 57336 32219467 A NORMAL'
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2015-11-10 08:56:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2015-11-10 08:56:59 [DEBUG   ] spawn_process: at 2015-11-10 08:56:59 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2015-11-10 08:57:05 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2015-11-10 08:57:26 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 08:57:46 [DEBUG   ] Finished updating ant stats in 1.230 seconds
2015-11-10 08:57:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 08:57:47 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 08:57:47 [DEBUG   ] Finished updating board flags in 1.738 seconds
2015-11-10 08:57:49 [DEBUG   ] Got command RPT from MCS: ref #535617
2015-11-10 08:57:49 [DEBUG   ] summary = NORMAL
2015-11-10 08:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535617  14 57336 32269071 A NORMALNORMAL'
2015-11-10 08:57:49 [DEBUG   ] Got command RPT from MCS: ref #535618
2015-11-10 08:57:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 08:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535618  48 57336 32269071 A NORMALSystem calibrated and operating normally'
2015-11-10 08:58:06 [DEBUG   ] Finished updating ant stats in 1.172 seconds
2015-11-10 08:58:26 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 08:58:46 [DEBUG   ] Finished updating ant stats in 1.029 seconds
2015-11-10 08:58:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 08:58:47 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 08:58:47 [DEBUG   ] Finished updating board flags in 1.592 seconds
2015-11-10 08:59:00 [DEBUG   ] Got command TBN from MCS: ref #535634
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '1', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '2', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] tbn = exited with status True, code 0x00
2015-11-10 08:59:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBN   535634   8 57336 32340295 A NORMAL'
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '3', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '4', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '5', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '6', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '7', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '8', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '9', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '10', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '11', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '12', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '13', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '14', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '15', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '16', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '17', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '18', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '19', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '20', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '21', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '22', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '23', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '24', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '25', '1447145941']
2015-11-10 08:59:00 [DEBUG   ] spawn_process: at 2015-11-10 08:59:00 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '25600000.0', '7', '18', '0', '26', '1447145941']
2015-11-10 08:59:00 [INFO    ] Finished spawning TBN in 0.059 s
2015-11-10 08:59:06 [DEBUG   ] Finished updating ant stats in 1.037 seconds
2015-11-10 08:59:26 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 08:59:46 [DEBUG   ] Finished updating ant stats in 1.048 seconds
2015-11-10 08:59:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2015-11-10 08:59:47 [DEBUG   ] Finished updating board flags in 1.618 seconds
2015-11-10 09:00:06 [DEBUG   ] Finished updating ant stats in 0.951 seconds
2015-11-10 09:00:26 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 09:00:29 [DEBUG   ] Got command RPT from MCS: ref #535638
2015-11-10 09:00:29 [DEBUG   ] summary = NORMAL
2015-11-10 09:00:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535638  14 57336 32429191 A NORMALNORMAL'
2015-11-10 09:00:46 [DEBUG   ] Finished updating ant stats in 0.850 seconds
2015-11-10 09:00:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 09:00:47 [DEBUG   ] Finished updating board flags in 1.620 seconds
2015-11-10 09:01:06 [DEBUG   ] Finished updating ant stats in 0.757 seconds
2015-11-10 09:01:26 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 09:01:46 [DEBUG   ] Finished updating ant stats in 0.643 seconds
2015-11-10 09:01:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 09:01:47 [DEBUG   ] Finished updating board flags in 1.598 seconds
2015-11-10 09:02:06 [DEBUG   ] Finished updating ant stats in 0.560 seconds
2015-11-10 09:02:27 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-10 09:02:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 09:02:47 [DEBUG   ] Finished updating ant stats in 1.252 seconds
2015-11-10 09:02:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:02:47 [DEBUG   ] Finished updating board flags in 1.448 seconds
2015-11-10 09:02:49 [DEBUG   ] Got command RPT from MCS: ref #535642
2015-11-10 09:02:49 [DEBUG   ] summary = NORMAL
2015-11-10 09:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535642  14 57336 32569071 A NORMALNORMAL'
2015-11-10 09:02:49 [DEBUG   ] Got command RPT from MCS: ref #535643
2015-11-10 09:02:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 09:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535643  48 57336 32569071 A NORMALSystem calibrated and operating normally'
2015-11-10 09:03:07 [DEBUG   ] Finished updating ant stats in 1.155 seconds
2015-11-10 09:03:27 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 09:03:47 [DEBUG   ] Finished updating ant stats in 1.043 seconds
2015-11-10 09:03:47 [DEBUG   ] Finished updating board flags in 1.446 seconds
2015-11-10 09:04:07 [DEBUG   ] Finished updating ant stats in 0.959 seconds
2015-11-10 09:04:27 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 09:04:47 [DEBUG   ] Finished updating ant stats in 0.847 seconds
2015-11-10 09:04:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:04:47 [DEBUG   ] Finished updating board flags in 1.461 seconds
2015-11-10 09:05:07 [DEBUG   ] Finished updating ant stats in 0.759 seconds
2015-11-10 09:05:27 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 09:05:29 [DEBUG   ] Got command RPT from MCS: ref #535660
2015-11-10 09:05:29 [DEBUG   ] summary = NORMAL
2015-11-10 09:05:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535660  14 57336 32729307 A NORMALNORMAL'
2015-11-10 09:05:47 [DEBUG   ] Finished updating ant stats in 0.643 seconds
2015-11-10 09:05:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:05:47 [DEBUG   ] Finished updating board flags in 1.453 seconds
2015-11-10 09:06:07 [DEBUG   ] Finished updating ant stats in 0.559 seconds
2015-11-10 09:06:28 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 09:06:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 09:06:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2015-11-10 09:06:48 [DEBUG   ] Finished updating ant stats in 1.251 seconds
2015-11-10 09:06:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:06:48 [DEBUG   ] Finished updating board flags in 2.260 seconds
2015-11-10 09:07:08 [DEBUG   ] Finished updating ant stats in 1.161 seconds
2015-11-10 09:07:28 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 09:07:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 09:07:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa34, chip 1
2015-11-10 09:07:48 [DEBUG   ] Finished updating ant stats in 1.043 seconds
2015-11-10 09:07:48 [DEBUG   ] Finished updating board flags in 2.215 seconds
2015-11-10 09:07:49 [DEBUG   ] Got command RPT from MCS: ref #535669
2015-11-10 09:07:49 [DEBUG   ] summary = NORMAL
2015-11-10 09:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535669  14 57336 32869079 A NORMALNORMAL'
2015-11-10 09:07:49 [DEBUG   ] Got command RPT from MCS: ref #535670
2015-11-10 09:07:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 09:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535670  48 57336 32869079 A NORMALSystem calibrated and operating normally'
2015-11-10 09:08:08 [DEBUG   ] Finished updating ant stats in 0.960 seconds
2015-11-10 09:08:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 09:08:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 09:08:48 [DEBUG   ] Finished updating ant stats in 0.843 seconds
2015-11-10 09:08:48 [DEBUG   ] Finished updating board flags in 2.184 seconds
2015-11-10 09:09:08 [DEBUG   ] Finished updating ant stats in 0.760 seconds
2015-11-10 09:09:28 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 09:09:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2015-11-10 09:09:48 [DEBUG   ] Finished updating ant stats in 0.643 seconds
2015-11-10 09:09:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 09:09:48 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:09:48 [DEBUG   ] Finished updating board flags in 2.018 seconds
2015-11-10 09:10:08 [DEBUG   ] Finished updating ant stats in 0.560 seconds
2015-11-10 09:10:29 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2015-11-10 09:10:29 [DEBUG   ] Got command RPT from MCS: ref #535689
2015-11-10 09:10:29 [DEBUG   ] summary = NORMAL
2015-11-10 09:10:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535689  14 57336 33029399 A NORMALNORMAL'
2015-11-10 09:10:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 09:10:48 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:10:48 [DEBUG   ] Finished updating board flags in 1.953 seconds
2015-11-10 09:10:49 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 09:11:09 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-10 09:11:29 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 09:11:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 09:11:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 09:11:48 [DEBUG   ] Finished updating board flags in 1.821 seconds
2015-11-10 09:11:49 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 09:12:09 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 09:12:29 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 09:12:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 09:12:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 09:12:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:12:48 [DEBUG   ] Finished updating board flags in 1.874 seconds
2015-11-10 09:12:49 [DEBUG   ] Got command RPT from MCS: ref #535693
2015-11-10 09:12:49 [DEBUG   ] summary = NORMAL
2015-11-10 09:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535693  14 57336 33169071 A NORMALNORMAL'
2015-11-10 09:12:49 [DEBUG   ] Got command RPT from MCS: ref #535694
2015-11-10 09:12:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 09:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535694  48 57336 33169071 A NORMALSystem calibrated and operating normally'
2015-11-10 09:12:49 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 09:13:09 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 09:13:29 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-10 09:13:48 [DEBUG   ] Finished updating board flags in 1.705 seconds
2015-11-10 09:13:49 [DEBUG   ] Finished updating ant stats in 0.794 seconds
2015-11-10 09:14:09 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 09:14:29 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 09:14:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 09:14:48 [DEBUG   ] Finished updating board flags in 1.602 seconds
2015-11-10 09:14:49 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 09:15:09 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2015-11-10 09:15:29 [DEBUG   ] Got command RPT from MCS: ref #535714
2015-11-10 09:15:29 [DEBUG   ] summary = NORMAL
2015-11-10 09:15:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535714  14 57336 33329503 A NORMALNORMAL'
2015-11-10 09:15:30 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 09:15:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 09:15:48 [DEBUG   ] Finished updating board flags in 1.575 seconds
2015-11-10 09:15:50 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-10 09:16:10 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 09:16:30 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 09:16:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa34, chip 1
2015-11-10 09:16:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 09:16:48 [DEBUG   ] Finished updating board flags in 1.421 seconds
2015-11-10 09:16:50 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 09:17:10 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 09:17:30 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 09:17:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 5
2015-11-10 09:17:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 09:17:48 [DEBUG   ] Finished updating board flags in 1.471 seconds
2015-11-10 09:17:49 [DEBUG   ] Got command RPT from MCS: ref #535723
2015-11-10 09:17:49 [DEBUG   ] summary = NORMAL
2015-11-10 09:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535723  14 57336 33469079 A NORMALNORMAL'
2015-11-10 09:17:49 [DEBUG   ] Got command RPT from MCS: ref #535724
2015-11-10 09:17:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 09:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535724  48 57336 33469079 A NORMALSystem calibrated and operating normally'
2015-11-10 09:17:50 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 09:18:10 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 09:18:30 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-10 09:18:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2015-11-10 09:18:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:18:48 [DEBUG   ] Finished updating board flags in 1.471 seconds
2015-11-10 09:18:50 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 09:19:10 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 09:19:30 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 09:19:48 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:19:48 [DEBUG   ] Finished updating board flags in 1.475 seconds
2015-11-10 09:19:50 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 09:20:10 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-10 09:20:29 [DEBUG   ] Got command RPT from MCS: ref #535740
2015-11-10 09:20:29 [DEBUG   ] summary = NORMAL
2015-11-10 09:20:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535740  14 57336 33629623 A NORMALNORMAL'
2015-11-10 09:20:30 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 09:20:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 09:20:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 5
2015-11-10 09:20:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:20:49 [DEBUG   ] Finished updating board flags in 2.221 seconds
2015-11-10 09:20:50 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 09:21:10 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2015-11-10 09:21:31 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 09:21:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:21:49 [DEBUG   ] Finished updating board flags in 2.155 seconds
2015-11-10 09:21:51 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-10 09:22:11 [DEBUG   ] Finished updating ant stats in 1.068 seconds
2015-11-10 09:22:31 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 09:22:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 09:22:49 [DEBUG   ] Got command RPT from MCS: ref #535744
2015-11-10 09:22:49 [DEBUG   ] summary = NORMAL
2015-11-10 09:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535744  14 57336 33769075 A NORMALNORMAL'
2015-11-10 09:22:49 [DEBUG   ] Got command RPT from MCS: ref #535745
2015-11-10 09:22:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 09:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535745  48 57336 33769076 A NORMALSystem calibrated and operating normally'
2015-11-10 09:22:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 09:22:49 [DEBUG   ] Finished updating board flags in 2.019 seconds
2015-11-10 09:22:51 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 09:23:11 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 09:23:31 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 09:23:49 [DEBUG   ] Finished updating board flags in 1.985 seconds
2015-11-10 09:23:51 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 09:24:11 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 09:24:31 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 09:24:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 09:24:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 09:24:49 [DEBUG   ] Finished updating board flags in 1.865 seconds
2015-11-10 09:24:51 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 09:25:11 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 09:25:29 [DEBUG   ] Got command RPT from MCS: ref #535765
2015-11-10 09:25:29 [DEBUG   ] summary = NORMAL
2015-11-10 09:25:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535765  14 57336 33929707 A NORMALNORMAL'
2015-11-10 09:25:31 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 09:25:49 [DEBUG   ] Finished updating board flags in 1.745 seconds
2015-11-10 09:25:52 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2015-11-10 09:26:12 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 09:26:32 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 09:26:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2015-11-10 09:26:49 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 09:26:49 [DEBUG   ] Finished updating board flags in 1.607 seconds
2015-11-10 09:26:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 09:27:12 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 09:27:32 [DEBUG   ] Finished updating ant stats in 1.159 seconds
2015-11-10 09:27:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2015-11-10 09:27:49 [DEBUG   ] Got command RPT from MCS: ref #535774
2015-11-10 09:27:49 [DEBUG   ] summary = NORMAL
2015-11-10 09:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535774  14 57336 34069079 A NORMALNORMAL'
2015-11-10 09:27:49 [DEBUG   ] Got command RPT from MCS: ref #535775
2015-11-10 09:27:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 09:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535775  48 57336 34069080 A NORMALSystem calibrated and operating normally'
2015-11-10 09:27:49 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 09:27:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:27:49 [DEBUG   ] Finished updating board flags in 1.610 seconds
2015-11-10 09:27:52 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 09:28:12 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 09:28:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 09:28:49 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 09:28:49 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:28:49 [DEBUG   ] Finished updating board flags in 1.588 seconds
2015-11-10 09:28:52 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 09:29:12 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 09:29:32 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 09:29:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 09:29:49 [DEBUG   ] Finished updating board flags in 1.409 seconds
2015-11-10 09:29:52 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 09:30:12 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 09:30:29 [DEBUG   ] Got command RPT from MCS: ref #535794
2015-11-10 09:30:29 [DEBUG   ] summary = NORMAL
2015-11-10 09:30:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535794  14 57336 34229803 A NORMALNORMAL'
2015-11-10 09:30:32 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 09:30:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 09:30:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 09:30:49 [DEBUG   ] Finished updating board flags in 1.389 seconds
2015-11-10 09:30:52 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 09:31:12 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2015-11-10 09:31:33 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 09:31:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 09:31:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 09:31:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa34, chip 1
2015-11-10 09:31:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 09:31:50 [DEBUG   ] Finished updating board flags in 2.211 seconds
2015-11-10 09:31:53 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-10 09:32:13 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-10 09:32:33 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 09:32:49 [DEBUG   ] Got command RPT from MCS: ref #535798
2015-11-10 09:32:49 [DEBUG   ] summary = NORMAL
2015-11-10 09:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535798  14 57336 34369088 A NORMALNORMAL'
2015-11-10 09:32:49 [DEBUG   ] Got command RPT from MCS: ref #535799
2015-11-10 09:32:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 09:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535799  48 57336 34369088 A NORMALSystem calibrated and operating normally'
2015-11-10 09:32:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 09:32:50 [DEBUG   ] Finished updating board flags in 2.189 seconds
2015-11-10 09:32:53 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2015-11-10 09:33:13 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 09:33:33 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 09:33:50 [DEBUG   ] Finished updating board flags in 2.019 seconds
2015-11-10 09:33:54 [DEBUG   ] Finished updating ant stats in 0.911 seconds
2015-11-10 09:34:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 09:34:34 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 09:34:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 09:34:50 [DEBUG   ] Finished updating board flags in 1.646 seconds
2015-11-10 09:34:54 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 09:35:14 [DEBUG   ] Finished updating ant stats in 0.795 seconds
2015-11-10 09:35:29 [DEBUG   ] Got command RPT from MCS: ref #535816
2015-11-10 09:35:29 [DEBUG   ] summary = NORMAL
2015-11-10 09:35:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535816  14 57336 34529895 A NORMALNORMAL'
2015-11-10 09:35:34 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 09:35:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 09:35:50 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:35:50 [DEBUG   ] Finished updating board flags in 1.611 seconds
2015-11-10 09:35:54 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2015-11-10 09:36:15 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 09:36:35 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-10 09:36:50 [DEBUG   ] Finished updating board flags in 1.610 seconds
2015-11-10 09:36:55 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 09:37:15 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 09:37:35 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 09:37:49 [DEBUG   ] Got command RPT from MCS: ref #535825
2015-11-10 09:37:49 [DEBUG   ] summary = NORMAL
2015-11-10 09:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535825  14 57336 34669083 A NORMALNORMAL'
2015-11-10 09:37:49 [DEBUG   ] Got command RPT from MCS: ref #535826
2015-11-10 09:37:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 09:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535826  48 57336 34669083 A NORMALSystem calibrated and operating normally'
2015-11-10 09:37:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 09:37:50 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:37:50 [DEBUG   ] Finished updating board flags in 1.611 seconds
2015-11-10 09:37:55 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 09:38:15 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 09:38:35 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 09:38:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 09:38:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2015-11-10 09:38:50 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 09:38:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 09:38:50 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:38:50 [DEBUG   ] Finished updating board flags in 1.580 seconds
2015-11-10 09:38:55 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 09:39:15 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 09:39:35 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 09:39:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 09:39:50 [DEBUG   ] Finished updating board flags in 1.487 seconds
2015-11-10 09:39:55 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 09:40:15 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 09:40:29 [DEBUG   ] Got command RPT from MCS: ref #535845
2015-11-10 09:40:29 [DEBUG   ] summary = NORMAL
2015-11-10 09:40:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535845  14 57336 34829979 A NORMALNORMAL'
2015-11-10 09:40:35 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 09:40:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 09:40:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2015-11-10 09:40:50 [DEBUG   ] Finished updating board flags in 1.564 seconds
2015-11-10 09:40:55 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 09:41:15 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 09:41:35 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 09:41:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:41:51 [DEBUG   ] Finished updating board flags in 2.247 seconds
2015-11-10 09:41:55 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 09:42:16 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 09:42:36 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-10 09:42:49 [DEBUG   ] Got command RPT from MCS: ref #535849
2015-11-10 09:42:49 [DEBUG   ] summary = NORMAL
2015-11-10 09:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535849  14 57336 34969079 A NORMALNORMAL'
2015-11-10 09:42:49 [DEBUG   ] Got command RPT from MCS: ref #535850
2015-11-10 09:42:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 09:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535850  48 57336 34969079 A NORMALSystem calibrated and operating normally'
2015-11-10 09:42:51 [DEBUG   ] Finished updating board flags in 2.111 seconds
2015-11-10 09:42:56 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 09:43:16 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 09:43:36 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 09:43:51 [DEBUG   ] Finished updating board flags in 2.006 seconds
2015-11-10 09:43:56 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 09:44:16 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 09:44:36 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 09:44:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 09:44:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 09:44:51 [DEBUG   ] Finished updating board flags in 2.016 seconds
2015-11-10 09:44:56 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2015-11-10 09:45:16 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 09:45:30 [DEBUG   ] Got command RPT from MCS: ref #535870
2015-11-10 09:45:30 [DEBUG   ] summary = NORMAL
2015-11-10 09:45:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535870  14 57336 35130079 A NORMALNORMAL'
2015-11-10 09:45:36 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 09:45:51 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:45:51 [DEBUG   ] Finished updating board flags in 2.005 seconds
2015-11-10 09:45:56 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 09:46:17 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-10 09:46:37 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 09:46:51 [DEBUG   ] Finished updating board flags in 1.977 seconds
2015-11-10 09:46:57 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-10 09:47:17 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 09:47:37 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2015-11-10 09:47:49 [DEBUG   ] Got command RPT from MCS: ref #535879
2015-11-10 09:47:49 [DEBUG   ] summary = NORMAL
2015-11-10 09:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535879  14 57336 35269091 A NORMALNORMAL'
2015-11-10 09:47:49 [DEBUG   ] Got command RPT from MCS: ref #535880
2015-11-10 09:47:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 09:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535880  48 57336 35269091 A NORMALSystem calibrated and operating normally'
2015-11-10 09:47:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 09:47:51 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:47:51 [DEBUG   ] Finished updating board flags in 1.819 seconds
2015-11-10 09:47:57 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-10 09:48:17 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-10 09:48:37 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 09:48:51 [DEBUG   ] Finished updating board flags in 1.792 seconds
2015-11-10 09:48:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 09:49:17 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2015-11-10 09:49:37 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 09:49:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 09:49:51 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 09:49:51 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:49:51 [DEBUG   ] Finished updating board flags in 1.604 seconds
2015-11-10 09:49:57 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 09:50:17 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 09:50:30 [DEBUG   ] Got command RPT from MCS: ref #535896
2015-11-10 09:50:30 [DEBUG   ] summary = NORMAL
2015-11-10 09:50:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535896  14 57336 35430175 A NORMALNORMAL'
2015-11-10 09:50:37 [DEBUG   ] Finished updating ant stats in 0.594 seconds
2015-11-10 09:50:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:50:51 [DEBUG   ] Finished updating board flags in 1.609 seconds
2015-11-10 09:50:58 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 09:51:18 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-10 09:51:38 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 09:51:50 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 09:51:51 [DEBUG   ] Finished updating board flags in 1.585 seconds
2015-11-10 09:51:58 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 09:52:18 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 09:52:38 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 09:52:49 [DEBUG   ] Got command RPT from MCS: ref #535900
2015-11-10 09:52:49 [DEBUG   ] summary = NORMAL
2015-11-10 09:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535900  14 57336 35569091 A NORMALNORMAL'
2015-11-10 09:52:49 [DEBUG   ] Got command RPT from MCS: ref #535901
2015-11-10 09:52:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 09:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535901  48 57336 35569091 A NORMALSystem calibrated and operating normally'
2015-11-10 09:52:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:52:51 [DEBUG   ] Finished updating board flags in 1.419 seconds
2015-11-10 09:52:58 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-10 09:53:18 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 09:53:38 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 09:53:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 09:53:51 [DEBUG   ] Finished updating board flags in 1.398 seconds
2015-11-10 09:53:58 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 09:54:18 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 09:54:38 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 09:54:52 [DEBUG   ] Finished updating board flags in 2.204 seconds
2015-11-10 09:54:58 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 09:55:18 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 09:55:30 [DEBUG   ] Got command RPT from MCS: ref #535921
2015-11-10 09:55:30 [DEBUG   ] summary = NORMAL
2015-11-10 09:55:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535921  14 57336 35730279 A NORMALNORMAL'
2015-11-10 09:55:38 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2015-11-10 09:55:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 09:55:52 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:55:52 [DEBUG   ] Finished updating board flags in 2.192 seconds
2015-11-10 09:55:59 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 09:56:19 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 09:56:39 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-10 09:56:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 09:56:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 09:56:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 09:56:52 [DEBUG   ] Finished updating board flags in 2.010 seconds
2015-11-10 09:56:59 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2015-11-10 09:57:19 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 09:57:39 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 09:57:49 [DEBUG   ] Got command RPT from MCS: ref #535930
2015-11-10 09:57:49 [DEBUG   ] summary = NORMAL
2015-11-10 09:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535930  14 57336 35869095 A NORMALNORMAL'
2015-11-10 09:57:49 [DEBUG   ] Got command RPT from MCS: ref #535931
2015-11-10 09:57:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 09:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535931  48 57336 35869095 A NORMALSystem calibrated and operating normally'
2015-11-10 09:57:52 [DEBUG   ] Finished updating board flags in 2.001 seconds
2015-11-10 09:57:59 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 09:58:19 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 09:58:39 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 09:58:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 09:58:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 09:58:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 09:58:52 [DEBUG   ] Finished updating board flags in 1.999 seconds
2015-11-10 09:58:59 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 09:59:19 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 09:59:40 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2015-11-10 09:59:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 09:59:52 [DEBUG   ] Finished updating board flags in 1.814 seconds
2015-11-10 10:00:00 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 10:00:20 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 10:00:30 [DEBUG   ] Got command RPT from MCS: ref #535950
2015-11-10 10:00:30 [DEBUG   ] summary = NORMAL
2015-11-10 10:00:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535950  14 57336 36030407 A NORMALNORMAL'
2015-11-10 10:00:40 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 10:00:52 [DEBUG   ] Finished updating board flags in 1.781 seconds
2015-11-10 10:01:00 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 10:01:20 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 10:01:40 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 10:01:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 10:01:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 10:01:52 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:01:52 [DEBUG   ] Finished updating board flags in 1.621 seconds
2015-11-10 10:02:00 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 10:02:20 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 10:02:40 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:02:49 [DEBUG   ] Got command RPT from MCS: ref #535954
2015-11-10 10:02:49 [DEBUG   ] summary = NORMAL
2015-11-10 10:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535954  14 57336 36169095 A NORMALNORMAL'
2015-11-10 10:02:49 [DEBUG   ] Got command RPT from MCS: ref #535955
2015-11-10 10:02:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 10:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535955  48 57336 36169095 A NORMALSystem calibrated and operating normally'
2015-11-10 10:02:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 10:02:52 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 10:02:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:02:52 [DEBUG   ] Finished updating board flags in 1.581 seconds
2015-11-10 10:03:00 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:03:20 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 10:03:40 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 10:03:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 10:03:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:03:52 [DEBUG   ] Finished updating board flags in 1.416 seconds
2015-11-10 10:04:00 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 10:04:20 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 10:04:41 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-10 10:04:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 10:04:52 [DEBUG   ] Finished updating board flags in 1.389 seconds
2015-11-10 10:05:01 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 10:05:21 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 10:05:30 [DEBUG   ] Got command RPT from MCS: ref #535972
2015-11-10 10:05:30 [DEBUG   ] summary = NORMAL
2015-11-10 10:05:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535972  14 57336 36330523 A NORMALNORMAL'
2015-11-10 10:05:41 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 10:05:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 10:05:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 10:05:52 [DEBUG   ] Finished updating board flags in 1.368 seconds
2015-11-10 10:06:01 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 10:06:21 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 10:06:41 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 10:06:53 [DEBUG   ] Finished updating board flags in 2.050 seconds
2015-11-10 10:07:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 10:07:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 10:07:41 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2015-11-10 10:07:49 [DEBUG   ] Got command RPT from MCS: ref #535981
2015-11-10 10:07:49 [DEBUG   ] summary = NORMAL
2015-11-10 10:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535981  14 57336 36469091 A NORMALNORMAL'
2015-11-10 10:07:49 [DEBUG   ] Got command RPT from MCS: ref #535982
2015-11-10 10:07:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 10:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   535982  48 57336 36469091 A NORMALSystem calibrated and operating normally'
2015-11-10 10:07:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 10:07:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 10:07:53 [DEBUG   ] Finished updating board flags in 2.014 seconds
2015-11-10 10:08:01 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 10:08:21 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 10:08:42 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 10:08:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 10:08:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:08:53 [DEBUG   ] Finished updating board flags in 1.981 seconds
2015-11-10 10:09:02 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-10 10:09:22 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 10:09:42 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 10:09:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2015-11-10 10:09:53 [DEBUG   ] Finished updating board flags in 1.809 seconds
2015-11-10 10:10:02 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-10 10:10:22 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:10:30 [DEBUG   ] Got command RPT from MCS: ref #536001
2015-11-10 10:10:30 [DEBUG   ] summary = NORMAL
2015-11-10 10:10:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536001  14 57336 36630627 A NORMALNORMAL'
2015-11-10 10:10:42 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 10:10:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 10:10:53 [DEBUG   ] Finished updating board flags in 1.811 seconds
2015-11-10 10:11:02 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:11:22 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2015-11-10 10:11:42 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 10:11:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2015-11-10 10:11:53 [DEBUG   ] Finished updating board flags in 1.784 seconds
2015-11-10 10:12:02 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 10:12:22 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 10:12:43 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 10:12:49 [DEBUG   ] Got command RPT from MCS: ref #536005
2015-11-10 10:12:49 [DEBUG   ] summary = NORMAL
2015-11-10 10:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536005  14 57336 36769095 A NORMALNORMAL'
2015-11-10 10:12:49 [DEBUG   ] Got command RPT from MCS: ref #536006
2015-11-10 10:12:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 10:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536006  48 57336 36769095 A NORMALSystem calibrated and operating normally'
2015-11-10 10:12:53 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:12:53 [DEBUG   ] Finished updating board flags in 1.613 seconds
2015-11-10 10:13:03 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-10 10:13:23 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 10:13:43 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 10:13:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 10:13:53 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:13:53 [DEBUG   ] Finished updating board flags in 1.607 seconds
2015-11-10 10:14:03 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 10:14:23 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:14:43 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 10:14:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:14:53 [DEBUG   ] Finished updating board flags in 1.590 seconds
2015-11-10 10:15:04 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 10:15:24 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 10:15:30 [DEBUG   ] Got command RPT from MCS: ref #536026
2015-11-10 10:15:30 [DEBUG   ] summary = NORMAL
2015-11-10 10:15:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536026  14 57336 36930711 A NORMALNORMAL'
2015-11-10 10:15:44 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 10:15:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:15:53 [DEBUG   ] Finished updating board flags in 1.406 seconds
2015-11-10 10:16:04 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 10:16:24 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 10:16:44 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 10:16:53 [DEBUG   ] Finished updating board flags in 1.385 seconds
2015-11-10 10:17:04 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 10:17:24 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 10:17:44 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 10:17:49 [DEBUG   ] Got command RPT from MCS: ref #536035
2015-11-10 10:17:49 [DEBUG   ] summary = NORMAL
2015-11-10 10:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536035  14 57336 37069091 A NORMALNORMAL'
2015-11-10 10:17:49 [DEBUG   ] Got command RPT from MCS: ref #536036
2015-11-10 10:17:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 10:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536036  48 57336 37069091 A NORMALSystem calibrated and operating normally'
2015-11-10 10:17:53 [DEBUG   ] Finished updating board flags in 1.372 seconds
2015-11-10 10:18:04 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-10 10:18:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:18:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:18:54 [DEBUG   ] Finished updating board flags in 2.037 seconds
2015-11-10 10:19:04 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 10:19:24 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 10:19:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 10:19:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 10:19:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 10:19:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:19:54 [DEBUG   ] Finished updating board flags in 2.013 seconds
2015-11-10 10:20:04 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 10:20:25 [DEBUG   ] Finished updating ant stats in 1.403 seconds
2015-11-10 10:20:30 [DEBUG   ] Got command RPT from MCS: ref #536052
2015-11-10 10:20:30 [DEBUG   ] summary = NORMAL
2015-11-10 10:20:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536052  14 57336 37230823 A NORMALNORMAL'
2015-11-10 10:20:45 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 10:20:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 10:20:54 [DEBUG   ] Finished updating board flags in 1.989 seconds
2015-11-10 10:21:05 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-10 10:21:25 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-10 10:21:45 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 10:21:54 [DEBUG   ] Finished updating board flags in 1.812 seconds
2015-11-10 10:22:05 [DEBUG   ] Finished updating ant stats in 0.994 seconds
2015-11-10 10:22:25 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 10:22:45 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:22:49 [DEBUG   ] Got command RPT from MCS: ref #536056
2015-11-10 10:22:49 [DEBUG   ] summary = NORMAL
2015-11-10 10:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536056  14 57336 37369099 A NORMALNORMAL'
2015-11-10 10:22:49 [DEBUG   ] Got command RPT from MCS: ref #536057
2015-11-10 10:22:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 10:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536057  48 57336 37369099 A NORMALSystem calibrated and operating normally'
2015-11-10 10:22:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2015-11-10 10:22:54 [DEBUG   ] Finished updating board flags in 1.795 seconds
2015-11-10 10:23:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 10:23:25 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2015-11-10 10:23:45 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 10:23:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 10:23:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 10:23:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:23:54 [DEBUG   ] Finished updating board flags in 1.605 seconds
2015-11-10 10:24:05 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 10:24:25 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 10:24:45 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2015-11-10 10:24:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 10:24:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 10:24:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:24:54 [DEBUG   ] Finished updating board flags in 1.625 seconds
2015-11-10 10:25:06 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2015-11-10 10:25:26 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-10 10:25:30 [DEBUG   ] Got command RPT from MCS: ref #536077
2015-11-10 10:25:30 [DEBUG   ] summary = NORMAL
2015-11-10 10:25:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536077  14 57336 37530903 A NORMALNORMAL'
2015-11-10 10:25:46 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 10:25:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 10:25:54 [DEBUG   ] Finished updating board flags in 1.566 seconds
2015-11-10 10:26:06 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 10:26:26 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 10:26:46 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 10:26:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 10:26:54 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:26:54 [DEBUG   ] Finished updating board flags in 1.440 seconds
2015-11-10 10:27:06 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 10:27:26 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 10:27:46 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 10:27:49 [DEBUG   ] Got command RPT from MCS: ref #536086
2015-11-10 10:27:49 [DEBUG   ] summary = NORMAL
2015-11-10 10:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536086  14 57336 37669099 A NORMALNORMAL'
2015-11-10 10:27:49 [DEBUG   ] Got command RPT from MCS: ref #536087
2015-11-10 10:27:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 10:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536087  48 57336 37669099 A NORMALSystem calibrated and operating normally'
2015-11-10 10:27:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:27:54 [DEBUG   ] Finished updating board flags in 1.426 seconds
2015-11-10 10:28:06 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 10:28:26 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 10:28:46 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 10:28:54 [WARNING ] MonitorFlags: VCCINT 0.976 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:28:54 [DEBUG   ] Finished updating board flags in 1.424 seconds
2015-11-10 10:29:06 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 10:29:26 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:29:46 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 10:29:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:29:54 [DEBUG   ] Finished updating board flags in 1.426 seconds
2015-11-10 10:30:06 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 10:30:26 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2015-11-10 10:30:30 [DEBUG   ] Got command RPT from MCS: ref #536106
2015-11-10 10:30:30 [DEBUG   ] summary = NORMAL
2015-11-10 10:30:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536106  14 57336 37830983 A NORMALNORMAL'
2015-11-10 10:30:47 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2015-11-10 10:30:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2015-11-10 10:30:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 10:30:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:30:54 [DEBUG   ] Finished updating board flags in 1.424 seconds
2015-11-10 10:31:07 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-10 10:31:27 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 10:31:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 10:31:54 [DEBUG   ] Finished updating board flags in 1.426 seconds
2015-11-10 10:32:07 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 10:32:27 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2015-11-10 10:32:47 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 10:32:49 [DEBUG   ] Got command RPT from MCS: ref #536110
2015-11-10 10:32:49 [DEBUG   ] summary = NORMAL
2015-11-10 10:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536110  14 57336 37969095 A NORMALNORMAL'
2015-11-10 10:32:49 [DEBUG   ] Got command RPT from MCS: ref #536111
2015-11-10 10:32:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 10:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536111  48 57336 37969095 A NORMALSystem calibrated and operating normally'
2015-11-10 10:32:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:32:54 [DEBUG   ] Finished updating board flags in 1.424 seconds
2015-11-10 10:33:07 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 10:33:27 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2015-11-10 10:33:47 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:33:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:33:54 [DEBUG   ] Finished updating board flags in 1.426 seconds
2015-11-10 10:34:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:34:27 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 10:34:47 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 10:34:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 10:34:54 [DEBUG   ] Finished updating board flags in 1.429 seconds
2015-11-10 10:35:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 10:35:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 10:35:31 [DEBUG   ] Got command RPT from MCS: ref #536128
2015-11-10 10:35:31 [DEBUG   ] summary = NORMAL
2015-11-10 10:35:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536128  14 57336 38131083 A NORMALNORMAL'
2015-11-10 10:35:47 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 10:35:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:35:55 [DEBUG   ] Finished updating board flags in 2.221 seconds
2015-11-10 10:36:08 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 10:36:28 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 10:36:48 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 10:36:55 [DEBUG   ] Finished updating board flags in 2.160 seconds
2015-11-10 10:37:08 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 10:37:28 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 10:37:48 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 10:37:49 [DEBUG   ] Got command RPT from MCS: ref #536137
2015-11-10 10:37:49 [DEBUG   ] summary = NORMAL
2015-11-10 10:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536137  14 57336 38269103 A NORMALNORMAL'
2015-11-10 10:37:49 [DEBUG   ] Got command RPT from MCS: ref #536138
2015-11-10 10:37:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 10:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536138  48 57336 38269103 A NORMALSystem calibrated and operating normally'
2015-11-10 10:37:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 10:37:55 [DEBUG   ] Finished updating board flags in 2.040 seconds
2015-11-10 10:38:08 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 10:38:28 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 10:38:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:38:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 10:38:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 10:38:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 10:38:55 [DEBUG   ] Finished updating board flags in 1.971 seconds
2015-11-10 10:39:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 10:39:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:39:48 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:39:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 10:39:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2015-11-10 10:39:55 [DEBUG   ] Finished updating board flags in 1.839 seconds
2015-11-10 10:40:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:40:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:40:31 [DEBUG   ] Got command RPT from MCS: ref #536157
2015-11-10 10:40:31 [DEBUG   ] summary = NORMAL
2015-11-10 10:40:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536157  14 57336 38431183 A NORMALNORMAL'
2015-11-10 10:40:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:40:55 [DEBUG   ] Finished updating board flags in 1.758 seconds
2015-11-10 10:41:08 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:41:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:41:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:41:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:41:55 [DEBUG   ] Finished updating board flags in 1.625 seconds
2015-11-10 10:42:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:42:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:42:48 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:42:49 [DEBUG   ] Got command RPT from MCS: ref #536161
2015-11-10 10:42:49 [DEBUG   ] summary = NORMAL
2015-11-10 10:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536161  14 57336 38569103 A NORMALNORMAL'
2015-11-10 10:42:49 [DEBUG   ] Got command RPT from MCS: ref #536162
2015-11-10 10:42:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 10:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536162  48 57336 38569103 A NORMALSystem calibrated and operating normally'
2015-11-10 10:42:55 [DEBUG   ] Finished updating board flags in 1.657 seconds
2015-11-10 10:43:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:43:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:43:48 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:43:55 [DEBUG   ] Finished updating board flags in 1.527 seconds
2015-11-10 10:44:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:44:28 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:44:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:44:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 10:44:55 [DEBUG   ] Finished updating board flags in 1.557 seconds
2015-11-10 10:45:08 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:45:28 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:45:31 [DEBUG   ] Got command RPT from MCS: ref #536182
2015-11-10 10:45:31 [DEBUG   ] summary = NORMAL
2015-11-10 10:45:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536182  14 57336 38731303 A NORMALNORMAL'
2015-11-10 10:45:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:45:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 10:45:55 [DEBUG   ] Finished updating board flags in 1.575 seconds
2015-11-10 10:46:08 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 10:46:28 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:46:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:46:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 10:46:56 [DEBUG   ] Finished updating board flags in 2.167 seconds
2015-11-10 10:47:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:47:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:47:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 10:47:49 [DEBUG   ] Got command RPT from MCS: ref #536191
2015-11-10 10:47:49 [DEBUG   ] summary = NORMAL
2015-11-10 10:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536191  14 57336 38869107 A NORMALNORMAL'
2015-11-10 10:47:49 [DEBUG   ] Got command RPT from MCS: ref #536192
2015-11-10 10:47:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 10:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536192  48 57336 38869107 A NORMALSystem calibrated and operating normally'
2015-11-10 10:47:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 10:47:56 [DEBUG   ] Finished updating board flags in 2.022 seconds
2015-11-10 10:48:08 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 10:48:28 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 10:48:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 10:48:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 10:48:56 [DEBUG   ] Finished updating board flags in 1.906 seconds
2015-11-10 10:49:08 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 10:49:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 10:49:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 10:49:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 10:49:56 [DEBUG   ] Finished updating board flags in 1.799 seconds
2015-11-10 10:50:08 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 10:50:28 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 10:50:31 [DEBUG   ] Got command RPT from MCS: ref #536208
2015-11-10 10:50:31 [DEBUG   ] summary = NORMAL
2015-11-10 10:50:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536208  14 57336 39031399 A NORMALNORMAL'
2015-11-10 10:50:49 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-10 10:50:56 [DEBUG   ] Finished updating board flags in 1.609 seconds
2015-11-10 10:51:09 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 10:51:29 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 10:51:49 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 10:51:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:51:56 [DEBUG   ] Finished updating board flags in 1.581 seconds
2015-11-10 10:52:09 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 10:52:29 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 10:52:49 [DEBUG   ] Got command RPT from MCS: ref #536212
2015-11-10 10:52:49 [DEBUG   ] summary = NORMAL
2015-11-10 10:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536212  14 57336 39169099 A NORMALNORMAL'
2015-11-10 10:52:49 [DEBUG   ] Got command RPT from MCS: ref #536213
2015-11-10 10:52:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 10:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536213  48 57336 39169099 A NORMALSystem calibrated and operating normally'
2015-11-10 10:52:49 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 10:52:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 10:52:56 [DEBUG   ] Finished updating board flags in 1.414 seconds
2015-11-10 10:53:09 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 10:53:29 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 10:53:49 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-10 10:53:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 10:53:56 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:53:56 [DEBUG   ] Finished updating board flags in 1.397 seconds
2015-11-10 10:54:09 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-10 10:54:29 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 10:54:49 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 10:54:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 10:54:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2015-11-10 10:54:57 [DEBUG   ] Finished updating board flags in 2.212 seconds
2015-11-10 10:55:09 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 10:55:29 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 10:55:31 [DEBUG   ] Got command RPT from MCS: ref #536233
2015-11-10 10:55:31 [DEBUG   ] summary = NORMAL
2015-11-10 10:55:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536233  14 57336 39331479 A NORMALNORMAL'
2015-11-10 10:55:49 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 10:55:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 10:55:57 [DEBUG   ] Finished updating board flags in 2.178 seconds
2015-11-10 10:56:09 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 10:56:29 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 10:56:50 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-10 10:56:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 10:56:57 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:56:57 [DEBUG   ] Finished updating board flags in 2.022 seconds
2015-11-10 10:57:10 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 10:57:30 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-10 10:57:49 [DEBUG   ] Got command RPT from MCS: ref #536242
2015-11-10 10:57:49 [DEBUG   ] summary = NORMAL
2015-11-10 10:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536242  14 57336 39469103 A NORMALNORMAL'
2015-11-10 10:57:49 [DEBUG   ] Got command RPT from MCS: ref #536243
2015-11-10 10:57:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 10:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536243  48 57336 39469103 A NORMALSystem calibrated and operating normally'
2015-11-10 10:57:50 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 10:57:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:57:57 [DEBUG   ] Finished updating board flags in 1.984 seconds
2015-11-10 10:58:10 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 10:58:30 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 10:58:50 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-10 10:58:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:58:57 [DEBUG   ] Finished updating board flags in 1.905 seconds
2015-11-10 10:59:10 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 10:59:30 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 10:59:50 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 10:59:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 10:59:57 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 10:59:57 [DEBUG   ] Finished updating board flags in 1.695 seconds
2015-11-10 11:00:10 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-10 11:00:30 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 11:00:31 [DEBUG   ] Got command RPT from MCS: ref #536262
2015-11-10 11:00:31 [DEBUG   ] summary = NORMAL
2015-11-10 11:00:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536262  14 57336 39631559 A NORMALNORMAL'
2015-11-10 11:00:50 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:00:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:00:57 [DEBUG   ] Finished updating board flags in 1.619 seconds
2015-11-10 11:01:10 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 11:01:30 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 11:01:51 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 11:01:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 11:01:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 11:01:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:01:57 [DEBUG   ] Finished updating board flags in 1.592 seconds
2015-11-10 11:02:11 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 11:02:31 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 11:02:49 [DEBUG   ] Got command RPT from MCS: ref #536266
2015-11-10 11:02:49 [DEBUG   ] summary = NORMAL
2015-11-10 11:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536266  14 57336 39769103 A NORMALNORMAL'
2015-11-10 11:02:49 [DEBUG   ] Got command RPT from MCS: ref #536267
2015-11-10 11:02:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 11:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536267  48 57336 39769103 A NORMALSystem calibrated and operating normally'
2015-11-10 11:02:51 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 11:02:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 11:02:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 11:02:57 [DEBUG   ] Finished updating board flags in 1.418 seconds
2015-11-10 11:03:11 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 11:03:31 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 11:03:51 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 11:03:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 11:03:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:03:57 [DEBUG   ] Finished updating board flags in 1.417 seconds
2015-11-10 11:04:11 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 11:04:31 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 11:04:51 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 11:04:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 11:04:57 [DEBUG   ] Finished updating board flags in 1.424 seconds
2015-11-10 11:05:11 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 11:05:31 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 11:05:31 [DEBUG   ] Got command RPT from MCS: ref #536284
2015-11-10 11:05:31 [DEBUG   ] summary = NORMAL
2015-11-10 11:05:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536284  14 57336 39931647 A NORMALNORMAL'
2015-11-10 11:05:51 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 11:05:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:05:57 [DEBUG   ] Finished updating board flags in 1.424 seconds
2015-11-10 11:06:11 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 11:06:32 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2015-11-10 11:06:52 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-10 11:06:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 11:06:57 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:06:57 [DEBUG   ] Finished updating board flags in 1.423 seconds
2015-11-10 11:07:12 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 11:07:32 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 11:07:49 [DEBUG   ] Got command RPT from MCS: ref #536293
2015-11-10 11:07:49 [DEBUG   ] summary = NORMAL
2015-11-10 11:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536293  14 57336 40069103 A NORMALNORMAL'
2015-11-10 11:07:49 [DEBUG   ] Got command RPT from MCS: ref #536294
2015-11-10 11:07:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 11:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536294  48 57336 40069103 A NORMALSystem calibrated and operating normally'
2015-11-10 11:07:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 11:07:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:07:57 [DEBUG   ] Finished updating board flags in 1.439 seconds
2015-11-10 11:08:12 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-10 11:08:32 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 11:08:52 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 11:08:57 [DEBUG   ] Finished updating board flags in 1.440 seconds
2015-11-10 11:09:12 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 11:09:32 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 11:09:52 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 11:09:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 11:09:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:09:57 [DEBUG   ] Finished updating board flags in 1.451 seconds
2015-11-10 11:10:12 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 11:10:31 [DEBUG   ] Got command RPT from MCS: ref #536313
2015-11-10 11:10:31 [DEBUG   ] summary = NORMAL
2015-11-10 11:10:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536313  14 57336 40231743 A NORMALNORMAL'
2015-11-10 11:10:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 11:10:52 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 11:10:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:10:58 [DEBUG   ] Finished updating board flags in 2.282 seconds
2015-11-10 11:11:12 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-10 11:11:32 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:11:52 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:11:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:11:58 [DEBUG   ] Finished updating board flags in 2.095 seconds
2015-11-10 11:12:12 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 11:12:32 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 11:12:49 [DEBUG   ] Got command RPT from MCS: ref #536317
2015-11-10 11:12:49 [DEBUG   ] summary = NORMAL
2015-11-10 11:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536317  14 57336 40369107 A NORMALNORMAL'
2015-11-10 11:12:49 [DEBUG   ] Got command RPT from MCS: ref #536318
2015-11-10 11:12:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 11:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536318  48 57336 40369107 A NORMALSystem calibrated and operating normally'
2015-11-10 11:12:52 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:12:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:12:58 [DEBUG   ] Finished updating board flags in 2.034 seconds
2015-11-10 11:13:12 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:13:32 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 11:13:52 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 11:13:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:13:58 [DEBUG   ] Finished updating board flags in 1.970 seconds
2015-11-10 11:14:12 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:14:32 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:14:52 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 11:14:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 11:14:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 11:14:58 [DEBUG   ] Finished updating board flags in 1.824 seconds
2015-11-10 11:15:12 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 11:15:31 [DEBUG   ] Got command RPT from MCS: ref #536338
2015-11-10 11:15:31 [DEBUG   ] summary = NORMAL
2015-11-10 11:15:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536338  14 57336 40531839 A NORMALNORMAL'
2015-11-10 11:15:32 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 11:15:52 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:15:58 [DEBUG   ] Finished updating board flags in 1.865 seconds
2015-11-10 11:16:12 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 11:16:32 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 11:16:52 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:16:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 11:16:58 [WARNING ] MonitorFlags: VCCINT 0.976 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:16:58 [DEBUG   ] Finished updating board flags in 1.809 seconds
2015-11-10 11:17:12 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:17:32 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 11:17:49 [DEBUG   ] Got command RPT from MCS: ref #536347
2015-11-10 11:17:49 [DEBUG   ] summary = NORMAL
2015-11-10 11:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536347  14 57336 40669107 A NORMALNORMAL'
2015-11-10 11:17:49 [DEBUG   ] Got command RPT from MCS: ref #536348
2015-11-10 11:17:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 11:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536348  48 57336 40669107 A NORMALSystem calibrated and operating normally'
2015-11-10 11:17:52 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 11:17:58 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:17:58 [DEBUG   ] Finished updating board flags in 1.763 seconds
2015-11-10 11:18:12 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:18:32 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:18:52 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 11:18:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:18:58 [DEBUG   ] Finished updating board flags in 1.584 seconds
2015-11-10 11:19:12 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 11:19:32 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:19:52 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:19:58 [DEBUG   ] Finished updating board flags in 1.453 seconds
2015-11-10 11:20:12 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 11:20:31 [DEBUG   ] Got command RPT from MCS: ref #536364
2015-11-10 11:20:31 [DEBUG   ] summary = NORMAL
2015-11-10 11:20:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536364  14 57336 40831939 A NORMALNORMAL'
2015-11-10 11:20:32 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 11:20:52 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 11:20:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 11:20:58 [DEBUG   ] Finished updating board flags in 1.457 seconds
2015-11-10 11:21:12 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 11:21:32 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 11:21:52 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 11:21:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2015-11-10 11:21:58 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:21:58 [DEBUG   ] Finished updating board flags in 1.459 seconds
2015-11-10 11:22:12 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 11:22:32 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 11:22:49 [DEBUG   ] Got command RPT from MCS: ref #536368
2015-11-10 11:22:49 [DEBUG   ] summary = NORMAL
2015-11-10 11:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536368  14 57336 40969107 A NORMALNORMAL'
2015-11-10 11:22:49 [DEBUG   ] Got command RPT from MCS: ref #536369
2015-11-10 11:22:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 11:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536369  48 57336 40969107 A NORMALSystem calibrated and operating normally'
2015-11-10 11:22:52 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 11:22:59 [DEBUG   ] Finished updating board flags in 1.600 seconds
2015-11-10 11:23:12 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 11:23:32 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 11:23:52 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 11:23:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 11:23:58 [DEBUG   ] Finished updating board flags in 1.343 seconds
2015-11-10 11:24:12 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 11:24:33 [DEBUG   ] Finished updating ant stats in 1.601 seconds
2015-11-10 11:24:52 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 11:24:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 11:24:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 11:24:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 11:24:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 11:24:59 [DEBUG   ] Finished updating board flags in 2.096 seconds
2015-11-10 11:25:12 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 11:25:32 [DEBUG   ] Got command RPT from MCS: ref #536389
2015-11-10 11:25:32 [DEBUG   ] summary = NORMAL
2015-11-10 11:25:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536389  14 57336 41132015 A NORMALNORMAL'
2015-11-10 11:25:32 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 11:25:52 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 11:25:59 [DEBUG   ] Finished updating board flags in 1.980 seconds
2015-11-10 11:26:12 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 11:26:32 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 11:26:53 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2015-11-10 11:26:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 11:26:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:26:59 [DEBUG   ] Finished updating board flags in 1.808 seconds
2015-11-10 11:27:13 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-10 11:27:33 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 11:27:49 [DEBUG   ] Got command RPT from MCS: ref #536398
2015-11-10 11:27:49 [DEBUG   ] summary = NORMAL
2015-11-10 11:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536398  14 57336 41269111 A NORMALNORMAL'
2015-11-10 11:27:49 [DEBUG   ] Got command RPT from MCS: ref #536399
2015-11-10 11:27:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 11:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536399  48 57336 41269111 A NORMALSystem calibrated and operating normally'
2015-11-10 11:27:53 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 11:27:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 11:27:59 [DEBUG   ] Finished updating board flags in 1.790 seconds
2015-11-10 11:28:13 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 11:28:33 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 11:28:53 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 11:28:59 [DEBUG   ] Finished updating board flags in 1.618 seconds
2015-11-10 11:29:13 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 11:29:33 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 11:29:53 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 11:29:59 [DEBUG   ] Finished updating board flags in 1.579 seconds
2015-11-10 11:30:13 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 11:30:32 [DEBUG   ] Got command RPT from MCS: ref #536418
2015-11-10 11:30:32 [DEBUG   ] summary = NORMAL
2015-11-10 11:30:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536418  14 57336 41432091 A NORMALNORMAL'
2015-11-10 11:30:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 11:30:53 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 11:30:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:30:59 [DEBUG   ] Finished updating board flags in 1.428 seconds
2015-11-10 11:31:14 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 11:31:34 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 11:31:54 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-10 11:31:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 11:31:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:31:59 [DEBUG   ] Finished updating board flags in 1.381 seconds
2015-11-10 11:32:14 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 11:32:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 11:32:49 [DEBUG   ] Got command RPT from MCS: ref #536422
2015-11-10 11:32:49 [DEBUG   ] summary = NORMAL
2015-11-10 11:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536422  14 57336 41569111 A NORMALNORMAL'
2015-11-10 11:32:49 [DEBUG   ] Got command RPT from MCS: ref #536423
2015-11-10 11:32:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 11:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536423  48 57336 41569111 A NORMALSystem calibrated and operating normally'
2015-11-10 11:32:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 11:32:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 11:33:00 [DEBUG   ] Finished updating board flags in 2.208 seconds
2015-11-10 11:33:14 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 11:33:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 11:33:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 11:33:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 11:34:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:34:00 [DEBUG   ] Finished updating board flags in 2.216 seconds
2015-11-10 11:34:14 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 11:34:34 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-10 11:34:54 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-10 11:35:00 [DEBUG   ] Finished updating board flags in 2.187 seconds
2015-11-10 11:35:14 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 11:35:32 [DEBUG   ] Got command RPT from MCS: ref #536440
2015-11-10 11:35:32 [DEBUG   ] summary = NORMAL
2015-11-10 11:35:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536440  14 57336 41732179 A NORMALNORMAL'
2015-11-10 11:35:34 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 11:35:54 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2015-11-10 11:36:00 [DEBUG   ] Finished updating board flags in 2.017 seconds
2015-11-10 11:36:15 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 11:36:35 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 11:36:55 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 11:36:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 11:37:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 11:37:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:37:00 [DEBUG   ] Finished updating board flags in 1.981 seconds
2015-11-10 11:37:15 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 11:37:35 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 11:37:49 [DEBUG   ] Got command RPT from MCS: ref #536449
2015-11-10 11:37:49 [DEBUG   ] summary = NORMAL
2015-11-10 11:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536449  14 57336 41869115 A NORMALNORMAL'
2015-11-10 11:37:49 [DEBUG   ] Got command RPT from MCS: ref #536450
2015-11-10 11:37:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 11:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536450  48 57336 41869115 A NORMALSystem calibrated and operating normally'
2015-11-10 11:37:55 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 11:37:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 11:38:00 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:38:00 [DEBUG   ] Finished updating board flags in 1.874 seconds
2015-11-10 11:38:15 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 11:38:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 11:38:55 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-10 11:39:00 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:39:00 [DEBUG   ] Finished updating board flags in 1.732 seconds
2015-11-10 11:39:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 11:39:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:39:55 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:39:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 11:39:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 11:40:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:40:00 [DEBUG   ] Finished updating board flags in 1.613 seconds
2015-11-10 11:40:15 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:40:32 [DEBUG   ] Got command RPT from MCS: ref #536469
2015-11-10 11:40:32 [DEBUG   ] summary = NORMAL
2015-11-10 11:40:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536469  14 57336 42032243 A NORMALNORMAL'
2015-11-10 11:40:35 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 11:40:55 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:40:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 11:41:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 1
2015-11-10 11:41:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:41:00 [DEBUG   ] Finished updating board flags in 1.636 seconds
2015-11-10 11:41:15 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:41:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:41:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 11:42:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 11:42:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:42:00 [DEBUG   ] Finished updating board flags in 1.578 seconds
2015-11-10 11:42:15 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:42:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:42:49 [DEBUG   ] Got command RPT from MCS: ref #536473
2015-11-10 11:42:49 [DEBUG   ] summary = NORMAL
2015-11-10 11:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536473  14 57336 42169111 A NORMALNORMAL'
2015-11-10 11:42:49 [DEBUG   ] Got command RPT from MCS: ref #536474
2015-11-10 11:42:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 11:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536474  48 57336 42169111 A NORMALSystem calibrated and operating normally'
2015-11-10 11:42:55 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:43:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 11:43:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 11:43:00 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:43:00 [DEBUG   ] Finished updating board flags in 1.565 seconds
2015-11-10 11:43:15 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:43:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:43:55 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:44:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 11:44:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 11:44:01 [DEBUG   ] Finished updating board flags in 1.581 seconds
2015-11-10 11:44:15 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:44:35 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 11:44:55 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 11:45:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 11:45:01 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:45:01 [DEBUG   ] Finished updating board flags in 2.139 seconds
2015-11-10 11:45:15 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 11:45:32 [DEBUG   ] Got command RPT from MCS: ref #536494
2015-11-10 11:45:32 [DEBUG   ] summary = NORMAL
2015-11-10 11:45:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536494  14 57336 42332327 A NORMALNORMAL'
2015-11-10 11:45:36 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 11:45:56 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 11:46:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 11:46:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:46:01 [DEBUG   ] Finished updating board flags in 1.900 seconds
2015-11-10 11:46:16 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 11:46:36 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 11:46:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 11:47:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 11:47:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 11:47:01 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 11:47:01 [DEBUG   ] Finished updating board flags in 1.823 seconds
2015-11-10 11:47:16 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 11:47:36 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 11:47:49 [DEBUG   ] Got command RPT from MCS: ref #536503
2015-11-10 11:47:49 [DEBUG   ] summary = NORMAL
2015-11-10 11:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536503  14 57336 42469115 A NORMALNORMAL'
2015-11-10 11:47:49 [DEBUG   ] Got command RPT from MCS: ref #536504
2015-11-10 11:47:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 11:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536504  48 57336 42469115 A NORMALSystem calibrated and operating normally'
2015-11-10 11:47:56 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 11:48:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 11:48:01 [DEBUG   ] Finished updating board flags in 1.800 seconds
2015-11-10 11:48:16 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 11:48:36 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 11:48:56 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 11:49:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 11:49:01 [DEBUG   ] Finished updating board flags in 1.620 seconds
2015-11-10 11:49:16 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 11:49:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 11:49:56 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 11:50:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:50:01 [DEBUG   ] Finished updating board flags in 1.648 seconds
2015-11-10 11:50:16 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 11:50:32 [DEBUG   ] Got command RPT from MCS: ref #536520
2015-11-10 11:50:32 [DEBUG   ] summary = NORMAL
2015-11-10 11:50:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536520  14 57336 42632411 A NORMALNORMAL'
2015-11-10 11:50:36 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 11:50:56 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 11:51:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 11:51:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 11:51:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 11:51:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:51:01 [DEBUG   ] Finished updating board flags in 1.518 seconds
2015-11-10 11:51:16 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-10 11:51:36 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:51:56 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:52:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 11:52:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 11:52:01 [DEBUG   ] Finished updating board flags in 1.454 seconds
2015-11-10 11:52:16 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2015-11-10 11:52:36 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 11:52:49 [DEBUG   ] Got command RPT from MCS: ref #536524
2015-11-10 11:52:49 [DEBUG   ] summary = NORMAL
2015-11-10 11:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536524  14 57336 42769111 A NORMALNORMAL'
2015-11-10 11:52:49 [DEBUG   ] Got command RPT from MCS: ref #536525
2015-11-10 11:52:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 11:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536525  48 57336 42769111 A NORMALSystem calibrated and operating normally'
2015-11-10 11:52:56 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 11:53:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 11:53:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:53:01 [DEBUG   ] Finished updating board flags in 1.454 seconds
2015-11-10 11:53:16 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 11:53:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 11:53:56 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 11:54:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 11:54:01 [DEBUG   ] Finished updating board flags in 1.506 seconds
2015-11-10 11:54:16 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 11:54:37 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 11:54:57 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 196000019
2015-11-10 11:55:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 196000019
2015-11-10 11:55:02 [WARNING ] MonitorFlags: VCCINT 0.976 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:55:02 [DEBUG   ] Finished updating board flags in 2.245 seconds
2015-11-10 11:55:17 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 11:55:32 [DEBUG   ] Got command RPT from MCS: ref #536545
2015-11-10 11:55:32 [DEBUG   ] summary = NORMAL
2015-11-10 11:55:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536545  14 57336 42932503 A NORMALNORMAL'
2015-11-10 11:55:37 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 11:55:57 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-10 11:56:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-10 11:56:02 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-10 11:56:02 [DEBUG   ] Finished updating board flags in 2.163 seconds
2015-11-10 11:56:17 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 11:56:37 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 11:56:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 11:57:02 [DEBUG   ] Finished updating board flags in 2.018 seconds
2015-11-10 11:57:17 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 11:57:37 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 11:57:49 [DEBUG   ] Got command RPT from MCS: ref #536554
2015-11-10 11:57:49 [DEBUG   ] summary = NORMAL
2015-11-10 11:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536554  14 57336 43069119 A NORMALNORMAL'
2015-11-10 11:57:49 [DEBUG   ] Got command RPT from MCS: ref #536555
2015-11-10 11:57:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 11:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536555  48 57336 43069120 A NORMALSystem calibrated and operating normally'
2015-11-10 11:57:57 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 11:58:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 11:58:02 [DEBUG   ] Finished updating board flags in 1.987 seconds
2015-11-10 11:58:17 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 11:58:37 [DEBUG   ] Finished updating ant stats in 0.977 seconds
2015-11-10 11:58:57 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 11:59:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 11:59:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 11:59:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 11:59:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2015-11-10 11:59:02 [DEBUG   ] Finished updating board flags in 1.813 seconds
2015-11-10 11:59:17 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 11:59:37 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 11:59:57 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 12:00:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 12:00:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 12:00:02 [DEBUG   ] Finished updating board flags in 1.776 seconds
2015-11-10 12:00:17 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 12:00:32 [DEBUG   ] Got command RPT from MCS: ref #536574
2015-11-10 12:00:32 [DEBUG   ] summary = NORMAL
2015-11-10 12:00:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536574  14 57336 43232583 A NORMALNORMAL'
2015-11-10 12:00:38 [DEBUG   ] Finished updating ant stats in 1.595 seconds
2015-11-10 12:00:58 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-10 12:01:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 12:01:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:01:02 [DEBUG   ] Finished updating board flags in 1.638 seconds
2015-11-10 12:01:17 [DEBUG   ] Finished updating ant stats in 0.406 seconds
2015-11-10 12:01:38 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2015-11-10 12:01:58 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 12:02:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 12:02:02 [DEBUG   ] Finished updating board flags in 1.571 seconds
2015-11-10 12:02:18 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2015-11-10 12:02:38 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 12:02:49 [DEBUG   ] Got command RPT from MCS: ref #536578
2015-11-10 12:02:49 [DEBUG   ] summary = NORMAL
2015-11-10 12:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536578  14 57336 43369123 A NORMALNORMAL'
2015-11-10 12:02:49 [DEBUG   ] Got command RPT from MCS: ref #536579
2015-11-10 12:02:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 12:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536579  48 57336 43369123 A NORMALSystem calibrated and operating normally'
2015-11-10 12:02:58 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 12:03:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 12:03:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:03:02 [DEBUG   ] Finished updating board flags in 1.416 seconds
2015-11-10 12:03:18 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 12:03:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 12:03:58 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 12:04:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 12:04:02 [DEBUG   ] Finished updating board flags in 1.390 seconds
2015-11-10 12:04:18 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-10 12:04:38 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 12:04:58 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 12:05:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:05:02 [DEBUG   ] Finished updating board flags in 1.371 seconds
2015-11-10 12:05:18 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 12:05:32 [DEBUG   ] Got command RPT from MCS: ref #536596
2015-11-10 12:05:32 [DEBUG   ] summary = NORMAL
2015-11-10 12:05:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536596  14 57336 43532691 A NORMALNORMAL'
2015-11-10 12:05:38 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 12:05:58 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 12:06:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 12:06:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2015-11-10 12:06:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:06:03 [DEBUG   ] Finished updating board flags in 2.044 seconds
2015-11-10 12:06:19 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-10 12:06:39 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 12:06:59 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 12:07:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 12:07:03 [DEBUG   ] Finished updating board flags in 1.990 seconds
2015-11-10 12:07:19 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2015-11-10 12:07:39 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:07:49 [DEBUG   ] Got command RPT from MCS: ref #536605
2015-11-10 12:07:49 [DEBUG   ] summary = NORMAL
2015-11-10 12:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536605  14 57336 43669119 A NORMALNORMAL'
2015-11-10 12:07:49 [DEBUG   ] Got command RPT from MCS: ref #536606
2015-11-10 12:07:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 12:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536606  48 57336 43669119 A NORMALSystem calibrated and operating normally'
2015-11-10 12:07:59 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 12:08:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2015-11-10 12:08:03 [DEBUG   ] Finished updating board flags in 1.811 seconds
2015-11-10 12:08:19 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 12:08:39 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 12:08:59 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 12:09:03 [DEBUG   ] Finished updating board flags in 1.793 seconds
2015-11-10 12:09:19 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2015-11-10 12:09:39 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 12:09:59 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2015-11-10 12:10:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 12:10:03 [DEBUG   ] Finished updating board flags in 1.604 seconds
2015-11-10 12:10:19 [DEBUG   ] Finished updating ant stats in 0.593 seconds
2015-11-10 12:10:32 [DEBUG   ] Got command RPT from MCS: ref #536625
2015-11-10 12:10:32 [DEBUG   ] summary = NORMAL
2015-11-10 12:10:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536625  14 57336 43832791 A NORMALNORMAL'
2015-11-10 12:10:40 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-10 12:11:00 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 12:11:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 12:11:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 12:11:03 [DEBUG   ] Finished updating board flags in 1.595 seconds
2015-11-10 12:11:20 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:11:40 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 12:12:00 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:12:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2015-11-10 12:12:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 12:12:03 [DEBUG   ] Finished updating board flags in 1.419 seconds
2015-11-10 12:12:20 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 12:12:40 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 12:12:49 [DEBUG   ] Got command RPT from MCS: ref #536629
2015-11-10 12:12:49 [DEBUG   ] summary = NORMAL
2015-11-10 12:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536629  14 57336 43969127 A NORMALNORMAL'
2015-11-10 12:12:49 [DEBUG   ] Got command RPT from MCS: ref #536630
2015-11-10 12:12:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 12:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536630  48 57336 43969127 A NORMALSystem calibrated and operating normally'
2015-11-10 12:13:00 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 12:13:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:13:03 [DEBUG   ] Finished updating board flags in 1.390 seconds
2015-11-10 12:13:20 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-10 12:13:40 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 12:14:00 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 12:14:03 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 12:14:03 [DEBUG   ] Finished updating board flags in 1.371 seconds
2015-11-10 12:14:20 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 12:14:40 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 12:15:00 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 12:15:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 12:15:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 12:15:04 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:15:04 [DEBUG   ] Finished updating board flags in 2.141 seconds
2015-11-10 12:15:20 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 12:15:32 [DEBUG   ] Got command RPT from MCS: ref #536650
2015-11-10 12:15:32 [DEBUG   ] summary = NORMAL
2015-11-10 12:15:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536650  14 57336 44132899 A NORMALNORMAL'
2015-11-10 12:15:40 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 12:16:00 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 12:16:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 12:16:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2015-11-10 12:16:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:16:04 [DEBUG   ] Finished updating board flags in 2.015 seconds
2015-11-10 12:16:20 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 12:16:40 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 12:17:00 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 12:17:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 12:17:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 12:17:04 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:17:04 [DEBUG   ] Finished updating board flags in 1.921 seconds
2015-11-10 12:17:20 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-10 12:17:40 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 12:17:49 [DEBUG   ] Got command RPT from MCS: ref #536659
2015-11-10 12:17:49 [DEBUG   ] summary = NORMAL
2015-11-10 12:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536659  14 57336 44269119 A NORMALNORMAL'
2015-11-10 12:17:49 [DEBUG   ] Got command RPT from MCS: ref #536660
2015-11-10 12:17:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 12:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536660  48 57336 44269119 A NORMALSystem calibrated and operating normally'
2015-11-10 12:18:00 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 12:18:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2015-11-10 12:18:04 [DEBUG   ] Finished updating board flags in 1.772 seconds
2015-11-10 12:18:20 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 12:18:40 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 12:19:00 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 12:19:04 [DEBUG   ] Finished updating board flags in 1.614 seconds
2015-11-10 12:19:20 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 12:19:41 [DEBUG   ] Finished updating ant stats in 1.391 seconds
2015-11-10 12:20:01 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-10 12:20:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2015-11-10 12:20:04 [DEBUG   ] Finished updating board flags in 1.580 seconds
2015-11-10 12:20:21 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:20:32 [DEBUG   ] Got command RPT from MCS: ref #536676
2015-11-10 12:20:32 [DEBUG   ] summary = NORMAL
2015-11-10 12:20:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536676  14 57336 44432995 A NORMALNORMAL'
2015-11-10 12:20:41 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 12:21:01 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 12:21:04 [DEBUG   ] Finished updating board flags in 1.426 seconds
2015-11-10 12:21:21 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 12:21:41 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 12:22:01 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 196000019
2015-11-10 12:22:03 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 196000019
2015-11-10 12:22:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 196000019
2015-11-10 12:22:04 [DEBUG   ] Finished updating board flags in 1.393 seconds
2015-11-10 12:22:21 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-10 12:22:41 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-10 12:22:49 [DEBUG   ] Got command RPT from MCS: ref #536680
2015-11-10 12:22:49 [DEBUG   ] summary = NORMAL
2015-11-10 12:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536680  14 57336 44569123 A NORMALNORMAL'
2015-11-10 12:22:49 [DEBUG   ] Got command RPT from MCS: ref #536681
2015-11-10 12:22:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 12:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536681  48 57336 44569123 A NORMALSystem calibrated and operating normally'
2015-11-10 12:23:01 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-10 12:23:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-10 12:23:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:23:05 [DEBUG   ] Finished updating board flags in 2.224 seconds
2015-11-10 12:23:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 12:23:41 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 12:24:01 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2015-11-10 12:24:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:24:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:24:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:24:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:24:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:24:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 12:24:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:24:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:24:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:24:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:24:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:24:05 [DEBUG   ] Finished updating board flags in 2.173 seconds
2015-11-10 12:24:21 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 12:24:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 12:25:01 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 12:25:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:25:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:25:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:25:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:25:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:25:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:25:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:25:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:25:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:25:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:25:05 [DEBUG   ] Finished updating board flags in 2.010 seconds
2015-11-10 12:25:21 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 12:25:33 [DEBUG   ] Got command RPT from MCS: ref #536701
2015-11-10 12:25:33 [DEBUG   ] summary = NORMAL
2015-11-10 12:25:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536701  14 57336 44733095 A NORMALNORMAL'
2015-11-10 12:25:41 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 12:26:01 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 12:26:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:26:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:26:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:26:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:26:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:26:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:26:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:26:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:26:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:26:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:26:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 12:26:05 [DEBUG   ] Finished updating board flags in 2.012 seconds
2015-11-10 12:26:21 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2015-11-10 12:26:42 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-10 12:27:02 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 12:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:27:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:27:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 12:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:27:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:27:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 12:27:05 [DEBUG   ] Finished updating board flags in 1.984 seconds
2015-11-10 12:27:22 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2015-11-10 12:27:42 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2015-11-10 12:27:49 [DEBUG   ] Got command RPT from MCS: ref #536710
2015-11-10 12:27:49 [DEBUG   ] summary = NORMAL
2015-11-10 12:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536710  14 57336 44869131 A NORMALNORMAL'
2015-11-10 12:27:49 [DEBUG   ] Got command RPT from MCS: ref #536711
2015-11-10 12:27:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 12:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536711  48 57336 44869131 A NORMALSystem calibrated and operating normally'
2015-11-10 12:28:02 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:28:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:28:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 12:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:28:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:28:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:28:05 [DEBUG   ] Finished updating board flags in 1.814 seconds
2015-11-10 12:28:22 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 12:28:42 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 12:29:02 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 12:29:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:29:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:29:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:29:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:29:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:29:04 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 12:29:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:29:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:29:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:29:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:29:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:29:05 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 12:29:05 [DEBUG   ] Finished updating board flags in 1.798 seconds
2015-11-10 12:29:22 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 12:29:42 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 12:30:02 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 12:30:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:30:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:30:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:30:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:30:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:30:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:30:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:30:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:30:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:30:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:30:05 [DEBUG   ] Finished updating board flags in 1.655 seconds
2015-11-10 12:30:22 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 12:30:33 [DEBUG   ] Got command RPT from MCS: ref #536730
2015-11-10 12:30:33 [DEBUG   ] summary = NORMAL
2015-11-10 12:30:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536730  14 57336 45033171 A NORMALNORMAL'
2015-11-10 12:30:43 [DEBUG   ] Finished updating ant stats in 1.595 seconds
2015-11-10 12:31:03 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 12:31:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:31:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:31:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:31:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:31:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:31:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:31:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:31:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:31:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:31:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:31:05 [DEBUG   ] Finished updating board flags in 1.618 seconds
2015-11-10 12:31:23 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2015-11-10 12:31:43 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-10 12:32:03 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 12:32:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:32:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:32:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:32:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:32:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:32:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:32:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:32:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:32:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:32:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:32:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 12:32:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:32:05 [DEBUG   ] Finished updating board flags in 1.529 seconds
2015-11-10 12:32:23 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:32:43 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:32:49 [DEBUG   ] Got command RPT from MCS: ref #536734
2015-11-10 12:32:49 [DEBUG   ] summary = NORMAL
2015-11-10 12:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536734  14 57336 45169123 A NORMALNORMAL'
2015-11-10 12:32:49 [DEBUG   ] Got command RPT from MCS: ref #536735
2015-11-10 12:32:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 12:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536735  48 57336 45169123 A NORMALSystem calibrated and operating normally'
2015-11-10 12:33:03 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:33:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:33:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:33:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:33:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:33:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:33:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:33:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:33:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:33:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:33:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:33:05 [DEBUG   ] Finished updating board flags in 1.456 seconds
2015-11-10 12:33:23 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 12:33:43 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 12:34:03 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 12:34:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:34:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:34:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:34:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:34:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:34:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:34:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:34:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:34:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:34:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:34:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 12:34:05 [DEBUG   ] Finished updating board flags in 1.450 seconds
2015-11-10 12:34:23 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2015-11-10 12:34:43 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 12:35:03 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 12:35:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:35:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:35:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:35:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:35:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:35:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:35:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:35:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:35:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:35:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:35:05 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 12:35:05 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:35:05 [DEBUG   ] Finished updating board flags in 1.451 seconds
2015-11-10 12:35:23 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 12:35:33 [DEBUG   ] Got command RPT from MCS: ref #536752
2015-11-10 12:35:33 [DEBUG   ] summary = NORMAL
2015-11-10 12:35:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536752  14 57336 45333275 A NORMALNORMAL'
2015-11-10 12:35:44 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2015-11-10 12:36:04 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 12:36:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:36:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:36:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:36:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:36:04 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:36:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 12:36:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 12:36:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:36:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:36:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:36:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:36:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:36:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 12:36:05 [DEBUG   ] Finished updating board flags in 1.465 seconds
2015-11-10 12:36:24 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:36:44 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 12:37:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 12:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:37:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:37:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:37:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:37:06 [DEBUG   ] Finished updating board flags in 2.237 seconds
2015-11-10 12:37:24 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:37:44 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:37:49 [DEBUG   ] Got command RPT from MCS: ref #536761
2015-11-10 12:37:49 [DEBUG   ] summary = NORMAL
2015-11-10 12:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536761  14 57336 45469131 A NORMALNORMAL'
2015-11-10 12:37:49 [DEBUG   ] Got command RPT from MCS: ref #536762
2015-11-10 12:37:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 12:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536762  48 57336 45469131 A NORMALSystem calibrated and operating normally'
2015-11-10 12:38:04 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:38:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:38:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 12:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:38:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:38:06 [DEBUG   ] Finished updating board flags in 2.148 seconds
2015-11-10 12:38:24 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 12:38:44 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:39:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 12:39:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:39:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:39:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:39:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:39:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:39:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:39:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:39:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:39:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:39:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:39:06 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:39:06 [DEBUG   ] Finished updating board flags in 2.080 seconds
2015-11-10 12:39:24 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 12:39:44 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 12:40:04 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 12:40:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:40:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:40:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:40:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:40:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:40:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 12:40:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:40:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:40:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:40:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:40:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:40:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 12:40:06 [DEBUG   ] Finished updating board flags in 2.028 seconds
2015-11-10 12:40:24 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:40:33 [DEBUG   ] Got command RPT from MCS: ref #536781
2015-11-10 12:40:33 [DEBUG   ] summary = NORMAL
2015-11-10 12:40:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536781  14 57336 45633399 A NORMALNORMAL'
2015-11-10 12:40:44 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 12:41:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 12:41:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:41:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:41:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:41:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:41:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:41:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 12:41:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:41:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:41:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:41:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:41:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:41:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 12:41:06 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:41:06 [DEBUG   ] Finished updating board flags in 1.910 seconds
2015-11-10 12:41:24 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:41:44 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:42:04 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 12:42:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:42:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:42:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:42:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:42:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:42:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:42:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:42:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:42:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:42:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:42:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:42:06 [DEBUG   ] Finished updating board flags in 1.834 seconds
2015-11-10 12:42:24 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 12:42:44 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 12:42:49 [DEBUG   ] Got command RPT from MCS: ref #536785
2015-11-10 12:42:49 [DEBUG   ] summary = NORMAL
2015-11-10 12:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536785  14 57336 45769131 A NORMALNORMAL'
2015-11-10 12:42:49 [DEBUG   ] Got command RPT from MCS: ref #536786
2015-11-10 12:42:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 12:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536786  48 57336 45769131 A NORMALSystem calibrated and operating normally'
2015-11-10 12:43:04 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:43:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:43:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:43:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:43:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:43:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:43:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 12:43:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:43:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:43:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:43:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:43:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:43:06 [DEBUG   ] Finished updating board flags in 1.771 seconds
2015-11-10 12:43:24 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:43:44 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 12:44:04 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2015-11-10 12:44:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999803
2015-11-10 12:44:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999803
2015-11-10 12:44:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999803
2015-11-10 12:44:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999803
2015-11-10 12:44:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999803
2015-11-10 12:44:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 12:44:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999803
2015-11-10 12:44:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999803
2015-11-10 12:44:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999803
2015-11-10 12:44:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999803
2015-11-10 12:44:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999803
2015-11-10 12:44:06 [DEBUG   ] Finished updating board flags in 1.619 seconds
2015-11-10 12:44:24 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:44:44 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:45:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-10 12:45:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-10 12:45:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:45:06 [DEBUG   ] Finished updating board flags in 1.673 seconds
2015-11-10 12:45:24 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2015-11-10 12:45:33 [DEBUG   ] Got command RPT from MCS: ref #536806
2015-11-10 12:45:33 [DEBUG   ] summary = NORMAL
2015-11-10 12:45:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536806  14 57336 45933519 A NORMALNORMAL'
2015-11-10 12:45:44 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 12:46:04 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-10 12:46:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-10 12:46:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-10 12:46:06 [DEBUG   ] Finished updating board flags in 1.513 seconds
2015-11-10 12:46:24 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 12:46:44 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 12:47:04 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 12:47:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:47:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:47:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:47:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:47:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:47:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 12:47:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 12:47:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 12:47:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:47:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:47:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:47:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:47:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:47:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 12:47:06 [DEBUG   ] Finished updating board flags in 1.431 seconds
2015-11-10 12:47:24 [DEBUG   ] Finished updating ant stats in 0.992 seconds
2015-11-10 12:47:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 12:47:49 [DEBUG   ] Got command RPT from MCS: ref #536815
2015-11-10 12:47:49 [DEBUG   ] summary = NORMAL
2015-11-10 12:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536815  14 57336 46069135 A NORMALNORMAL'
2015-11-10 12:47:49 [DEBUG   ] Got command RPT from MCS: ref #536816
2015-11-10 12:47:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 12:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536816  48 57336 46069135 A NORMALSystem calibrated and operating normally'
2015-11-10 12:48:04 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 12:48:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:48:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:48:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:48:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:48:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:48:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:48:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:48:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:48:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:48:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:48:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 12:48:06 [DEBUG   ] Finished updating board flags in 1.428 seconds
2015-11-10 12:48:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 12:48:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 12:49:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 12:49:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:49:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:49:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:49:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:49:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:49:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:49:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:49:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:49:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:49:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:49:06 [DEBUG   ] Finished updating board flags in 1.517 seconds
2015-11-10 12:49:24 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 12:49:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 12:50:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 12:50:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:50:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:50:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:50:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:50:05 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:50:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:50:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:50:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:50:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:50:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:50:06 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:50:06 [DEBUG   ] Finished updating board flags in 1.530 seconds
2015-11-10 12:50:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 12:50:33 [DEBUG   ] Got command RPT from MCS: ref #536832
2015-11-10 12:50:33 [DEBUG   ] summary = NORMAL
2015-11-10 12:50:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536832  14 57336 46233611 A NORMALNORMAL'
2015-11-10 12:50:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 12:51:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 12:51:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:51:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:51:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:51:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:51:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:51:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:51:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:51:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:51:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:51:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:51:07 [DEBUG   ] Finished updating board flags in 2.136 seconds
2015-11-10 12:51:24 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 12:51:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 12:52:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 12:52:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:52:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:52:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:52:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:52:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:52:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:52:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:52:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:52:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:52:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:52:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 12:52:07 [DEBUG   ] Finished updating board flags in 2.046 seconds
2015-11-10 12:52:24 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 12:52:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 12:52:49 [DEBUG   ] Got command RPT from MCS: ref #536836
2015-11-10 12:52:49 [DEBUG   ] summary = NORMAL
2015-11-10 12:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536836  14 57336 46369131 A NORMALNORMAL'
2015-11-10 12:52:49 [DEBUG   ] Got command RPT from MCS: ref #536837
2015-11-10 12:52:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 12:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536837  48 57336 46369131 A NORMALSystem calibrated and operating normally'
2015-11-10 12:53:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 12:53:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:53:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:53:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:53:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:53:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:53:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 12:53:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:53:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:53:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:53:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:53:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:53:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:53:07 [DEBUG   ] Finished updating board flags in 2.009 seconds
2015-11-10 12:53:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 12:53:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 12:54:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 12:54:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:54:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:54:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:54:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:54:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:54:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:54:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:54:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:54:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:54:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:54:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:54:07 [DEBUG   ] Finished updating board flags in 1.931 seconds
2015-11-10 12:54:24 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2015-11-10 12:54:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 12:55:04 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 12:55:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:55:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:55:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:55:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:55:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:55:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:55:07 [DEBUG   ] Finished updating board flags in 1.822 seconds
2015-11-10 12:55:25 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2015-11-10 12:55:33 [DEBUG   ] Got command RPT from MCS: ref #536857
2015-11-10 12:55:33 [DEBUG   ] summary = NORMAL
2015-11-10 12:55:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536857  14 57336 46533727 A NORMALNORMAL'
2015-11-10 12:55:45 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-10 12:56:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 12:56:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:56:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:56:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:56:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:56:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:56:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 12:56:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 12:56:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2015-11-10 12:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:56:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:56:07 [DEBUG   ] Finished updating board flags in 1.782 seconds
2015-11-10 12:56:25 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2015-11-10 12:56:45 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-10 12:57:05 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 12:57:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:57:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:57:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:57:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:57:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:57:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 12:57:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:57:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:57:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:57:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:57:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:57:07 [DEBUG   ] Finished updating board flags in 1.574 seconds
2015-11-10 12:57:25 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 12:57:45 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2015-11-10 12:57:49 [DEBUG   ] Got command RPT from MCS: ref #536866
2015-11-10 12:57:49 [DEBUG   ] summary = NORMAL
2015-11-10 12:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536866  14 57336 46669139 A NORMALNORMAL'
2015-11-10 12:57:49 [DEBUG   ] Got command RPT from MCS: ref #536867
2015-11-10 12:57:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 12:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536867  48 57336 46669139 A NORMALSystem calibrated and operating normally'
2015-11-10 12:58:05 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 12:58:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:58:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:58:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:58:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:58:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:58:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:58:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:58:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:58:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:58:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:58:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:58:07 [DEBUG   ] Finished updating board flags in 1.416 seconds
2015-11-10 12:58:25 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2015-11-10 12:58:45 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 12:59:05 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2015-11-10 12:59:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 12:59:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 12:59:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 12:59:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 12:59:06 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 12:59:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 12:59:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 12:59:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 12:59:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 12:59:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 12:59:07 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 12:59:07 [DEBUG   ] Finished updating board flags in 1.396 seconds
2015-11-10 12:59:25 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2015-11-10 12:59:46 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 13:00:06 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2015-11-10 13:00:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:00:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:00:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:00:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:00:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:00:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:00:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:00:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:00:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:00:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:00:08 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 13:00:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 13:00:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 13:00:08 [DEBUG   ] Finished updating board flags in 2.206 seconds
2015-11-10 13:00:26 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2015-11-10 13:00:33 [DEBUG   ] Got command RPT from MCS: ref #536886
2015-11-10 13:00:33 [DEBUG   ] summary = NORMAL
2015-11-10 13:00:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536886  14 57336 46833847 A NORMALNORMAL'
2015-11-10 13:00:46 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 13:01:06 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 13:01:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:01:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:01:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:01:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:01:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:01:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2015-11-10 13:01:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2015-11-10 13:01:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:01:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:01:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:01:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:01:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:01:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2015-11-10 13:01:08 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 13:01:08 [DEBUG   ] Finished updating board flags in 2.192 seconds
2015-11-10 13:01:26 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2015-11-10 13:01:46 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 13:02:06 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-10 13:02:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:02:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:02:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:02:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:02:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:02:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:02:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:02:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:02:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:02:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:02:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 13:02:08 [DEBUG   ] Finished updating board flags in 2.016 seconds
2015-11-10 13:02:26 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2015-11-10 13:02:46 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 13:02:49 [DEBUG   ] Got command RPT from MCS: ref #536890
2015-11-10 13:02:49 [DEBUG   ] summary = NORMAL
2015-11-10 13:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536890  14 57336 46969135 A NORMALNORMAL'
2015-11-10 13:02:49 [DEBUG   ] Got command RPT from MCS: ref #536891
2015-11-10 13:02:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 13:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536891  48 57336 46969135 A NORMALSystem calibrated and operating normally'
2015-11-10 13:03:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2015-11-10 13:03:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:03:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:03:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:03:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:03:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:03:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2015-11-10 13:03:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:03:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:03:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:03:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:03:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:03:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 13:03:08 [DEBUG   ] Finished updating board flags in 1.988 seconds
2015-11-10 13:03:27 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2015-11-10 13:03:47 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2015-11-10 13:04:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:04:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:04:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:04:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:04:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:04:07 [DEBUG   ] Finished updating ant stats in 1.255 seconds
2015-11-10 13:04:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:04:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:04:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:04:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:04:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:04:08 [DEBUG   ] Finished updating board flags in 1.815 seconds
2015-11-10 13:04:27 [DEBUG   ] Finished updating ant stats in 1.158 seconds
2015-11-10 13:04:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 13:05:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:05:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:05:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:05:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:05:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:05:07 [DEBUG   ] Finished updating ant stats in 1.047 seconds
2015-11-10 13:05:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:05:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:05:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:05:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:05:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:05:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2015-11-10 13:05:08 [DEBUG   ] Finished updating board flags in 1.783 seconds
2015-11-10 13:05:27 [DEBUG   ] Finished updating ant stats in 0.958 seconds
2015-11-10 13:05:33 [DEBUG   ] Got command RPT from MCS: ref #536908
2015-11-10 13:05:33 [DEBUG   ] summary = NORMAL
2015-11-10 13:05:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536908  14 57336 47133943 A NORMALNORMAL'
2015-11-10 13:05:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 13:06:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:06:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:06:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:06:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:06:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:06:07 [DEBUG   ] Finished updating ant stats in 0.837 seconds
2015-11-10 13:06:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:06:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:06:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:06:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:06:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:06:08 [DEBUG   ] Finished updating board flags in 1.621 seconds
2015-11-10 13:06:27 [DEBUG   ] Finished updating ant stats in 0.755 seconds
2015-11-10 13:06:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 13:07:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:07:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:07:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:07:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:07:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:07:07 [DEBUG   ] Finished updating ant stats in 0.649 seconds
2015-11-10 13:07:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:07:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:07:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:07:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:07:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:07:08 [DEBUG   ] Finished updating board flags in 1.599 seconds
2015-11-10 13:07:27 [DEBUG   ] Finished updating ant stats in 0.555 seconds
2015-11-10 13:07:48 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2015-11-10 13:07:49 [DEBUG   ] Got command RPT from MCS: ref #536917
2015-11-10 13:07:49 [DEBUG   ] summary = NORMAL
2015-11-10 13:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536917  14 57336 47269131 A NORMALNORMAL'
2015-11-10 13:07:49 [DEBUG   ] Got command RPT from MCS: ref #536918
2015-11-10 13:07:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 13:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536918  48 57336 47269131 A NORMALSystem calibrated and operating normally'
2015-11-10 13:08:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:08:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:08:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:08:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:08:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:08:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:08:08 [DEBUG   ] Finished updating ant stats in 1.249 seconds
2015-11-10 13:08:08 [DEBUG   ] Finished updating board flags in 1.461 seconds
2015-11-10 13:08:28 [DEBUG   ] Finished updating ant stats in 1.159 seconds
2015-11-10 13:08:48 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2015-11-10 13:09:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:09:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:09:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:09:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:09:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:09:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:09:08 [DEBUG   ] Finished updating ant stats in 1.044 seconds
2015-11-10 13:09:08 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 13:09:08 [DEBUG   ] Finished updating board flags in 1.458 seconds
2015-11-10 13:09:28 [DEBUG   ] Finished updating ant stats in 0.957 seconds
2015-11-10 13:09:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 13:10:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:10:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:10:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:10:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:10:07 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:10:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:10:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:10:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:10:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:10:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:10:08 [DEBUG   ] Finished updating ant stats in 0.845 seconds
2015-11-10 13:10:08 [DEBUG   ] Finished updating board flags in 1.495 seconds
2015-11-10 13:10:28 [DEBUG   ] Finished updating ant stats in 0.756 seconds
2015-11-10 13:10:34 [DEBUG   ] Got command RPT from MCS: ref #536937
2015-11-10 13:10:34 [DEBUG   ] summary = NORMAL
2015-11-10 13:10:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536937  14 57336 47434047 A NORMALNORMAL'
2015-11-10 13:10:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2015-11-10 13:11:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:11:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:11:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:11:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:11:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:11:08 [DEBUG   ] Finished updating ant stats in 0.649 seconds
2015-11-10 13:11:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 13:11:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:11:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:11:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:11:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:11:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:11:09 [DEBUG   ] Finished updating board flags in 2.191 seconds
2015-11-10 13:11:28 [DEBUG   ] Finished updating ant stats in 0.554 seconds
2015-11-10 13:11:49 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2015-11-10 13:12:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:12:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:12:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:12:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:12:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:12:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2015-11-10 13:12:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:12:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:12:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:12:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:12:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:12:09 [DEBUG   ] Finished updating ant stats in 1.251 seconds
2015-11-10 13:12:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 13:12:09 [DEBUG   ] Finished updating board flags in 2.050 seconds
2015-11-10 13:12:29 [DEBUG   ] Finished updating ant stats in 1.157 seconds
2015-11-10 13:12:49 [DEBUG   ] Got command RPT from MCS: ref #536941
2015-11-10 13:12:49 [DEBUG   ] summary = NORMAL
2015-11-10 13:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536941  14 57336 47569139 A NORMALNORMAL'
2015-11-10 13:12:49 [DEBUG   ] Got command RPT from MCS: ref #536942
2015-11-10 13:12:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 13:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536942  48 57336 47569139 A NORMALSystem calibrated and operating normally'
2015-11-10 13:12:49 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2015-11-10 13:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:13:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:13:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 13:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:13:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:13:09 [DEBUG   ] Finished updating ant stats in 1.044 seconds
2015-11-10 13:13:09 [DEBUG   ] Finished updating board flags in 2.023 seconds
2015-11-10 13:13:29 [DEBUG   ] Finished updating ant stats in 0.958 seconds
2015-11-10 13:13:49 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 13:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:14:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:14:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:14:09 [DEBUG   ] Finished updating ant stats in 0.647 seconds
2015-11-10 13:14:09 [DEBUG   ] Finished updating board flags in 1.992 seconds
2015-11-10 13:14:29 [DEBUG   ] Finished updating ant stats in 0.561 seconds
2015-11-10 13:14:50 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2015-11-10 13:15:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:15:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:15:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:15:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:15:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:15:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:15:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:15:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:15:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:15:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:15:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 13:15:09 [DEBUG   ] Finished updating board flags in 1.800 seconds
2015-11-10 13:15:10 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2015-11-10 13:15:30 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2015-11-10 13:15:34 [DEBUG   ] Got command RPT from MCS: ref #536962
2015-11-10 13:15:34 [DEBUG   ] summary = NORMAL
2015-11-10 13:15:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536962  14 57336 47734131 A NORMALNORMAL'
2015-11-10 13:15:50 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2015-11-10 13:16:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:16:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:16:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:16:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:16:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:16:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 13:16:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:16:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:16:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:16:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:16:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:16:09 [DEBUG   ] Finished updating board flags in 1.573 seconds
2015-11-10 13:16:10 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2015-11-10 13:16:30 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2015-11-10 13:16:50 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2015-11-10 13:17:01 [DEBUG   ] Got command STP from MCS: ref #536968
2015-11-10 13:17:01 [INFO    ] Finished processing BAM request
2015-11-10 13:17:01 [INFO    ] Shutdown the adder chain on beam #2
2015-11-10 13:17:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   536968   8 57336 47821135 A NORMAL'
2015-11-10 13:17:01 [DEBUG   ] Got command STP from MCS: ref #536970
2015-11-10 13:17:01 [INFO    ] Finished processing BAM request
2015-11-10 13:17:01 [INFO    ] Shutdown the adder chain on beam #4
2015-11-10 13:17:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   536970   8 57336 47821139 A NORMAL'
2015-11-10 13:17:01 [DEBUG   ] Got command STP from MCS: ref #536969
2015-11-10 13:17:01 [INFO    ] Finished processing BAM request
2015-11-10 13:17:01 [INFO    ] Shutdown the adder chain on beam #3
2015-11-10 13:17:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   536969   8 57336 47821143 A NORMAL'
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2015-11-10 13:17:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2015-11-10 13:17:02 [DEBUG   ] spawn_process: at 2015-11-10 13:17:02 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2015-11-10 13:17:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:17:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:17:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:17:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:17:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:17:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:17:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:17:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:17:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:17:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:17:09 [DEBUG   ] Finished updating board flags in 1.429 seconds
2015-11-10 13:17:10 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2015-11-10 13:17:30 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-10 13:17:49 [DEBUG   ] Got command RPT from MCS: ref #536974
2015-11-10 13:17:49 [DEBUG   ] summary = NORMAL
2015-11-10 13:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536974  14 57336 47869135 A NORMALNORMAL'
2015-11-10 13:17:49 [DEBUG   ] Got command RPT from MCS: ref #536975
2015-11-10 13:17:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 13:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536975  48 57336 47869135 A NORMALSystem calibrated and operating normally'
2015-11-10 13:17:50 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2015-11-10 13:18:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:18:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:18:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:18:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:18:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:18:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:18:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:18:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:18:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:18:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:18:09 [DEBUG   ] Finished updating board flags in 1.446 seconds
2015-11-10 13:18:10 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2015-11-10 13:18:30 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 13:18:50 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2015-11-10 13:19:00 [DEBUG   ] Got command INI from MCS: ref #536991
2015-11-10 13:19:00 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2015-11-10 13:19:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   536991   8 57336 47940311 ABOOTING'
2015-11-10 13:19:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 1 with 195999705
2015-11-10 13:19:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 2 with 195999705
2015-11-10 13:19:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 3 with 195999705
2015-11-10 13:19:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 4 with 195999705
2015-11-10 13:19:08 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa16, chip 5 with 195999705
2015-11-10 13:19:09 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2015-11-10 13:19:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 1 with 195999705
2015-11-10 13:19:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 2 with 195999705
2015-11-10 13:19:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 3 with 195999705
2015-11-10 13:19:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 4 with 195999705
2015-11-10 13:19:09 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa34, chip 5 with 195999705
2015-11-10 13:19:09 [DEBUG   ] Finished updating board flags in 1.444 seconds
2015-11-10 13:19:24 [INFO    ] -------------------
2015-11-10 13:19:24 [INFO    ]  LWA board status
2015-11-10 13:19:24 [INFO    ] -------------------
2015-11-10 13:19:24 [INFO    ] DP 1: 2 lwa27
2015-11-10 13:19:24 [INFO    ] DP 1: 3 lwa24
2015-11-10 13:19:24 [INFO    ] DP 1: 4 lwa23
2015-11-10 13:19:24 [INFO    ] DP 1: 5 lwa18
2015-11-10 13:19:24 [INFO    ] DP 1: 6 lwa10
2015-11-10 13:19:24 [INFO    ] DP 1: 7 lwa32
2015-11-10 13:19:24 [INFO    ] DP 1: 8 lwa11
2015-11-10 13:19:24 [INFO    ] DP 1: 9 lwa30
2015-11-10 13:19:24 [INFO    ] DP 1: 10 lwa12
2015-11-10 13:19:24 [INFO    ] DP 1: 11 lwa8
2015-11-10 13:19:24 [INFO    ] DP 1: 12 lwa13
2015-11-10 13:19:24 [INFO    ] DP 1: 13 lwa26
2015-11-10 13:19:24 [INFO    ] DP 1: 14 lwa14
2015-11-10 13:19:24 [INFO    ] DP 1: 16 lwa22
2015-11-10 13:19:24 [INFO    ] DP 1: 17 lwa20
2015-11-10 13:19:24 [INFO    ] DP 1: 18 lwa1
2015-11-10 13:19:24 [INFO    ] DP 1: 19 lwa5
2015-11-10 13:19:24 [INFO    ] DP 1: 20 lwa25
2015-11-10 13:19:24 [INFO    ] DP 1: 21 lwa35
2015-11-10 13:19:24 [INFO    ] DP 1: 22 lwa19
2015-11-10 13:19:24 [INFO    ] DP 1: 23 lwa21
2015-11-10 13:19:24 [INFO    ] DP 1: 24 lwa6
2015-11-10 13:19:24 [INFO    ] DP 1: 25 lwa7
2015-11-10 13:19:24 [INFO    ] DP 1: 26 lwa33
2015-11-10 13:19:24 [INFO    ] DP 1: 27 lwa15
2015-11-10 13:19:24 [INFO    ] DP 1: 28 lwa28
2015-11-10 13:19:24 [INFO    ] DP 2: 1 lwa16
2015-11-10 13:19:24 [INFO    ] DP 2: 15 lwa34
2015-11-10 13:19:24 [INFO    ] Total of 28 boards
2015-11-10 13:19:24 [INFO    ] Total of 260 stands
2015-11-10 13:19:24 [INFO    ] -------------------
2015-11-10 13:19:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19020
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19021
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19022
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19023
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19024
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19025
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19026
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19027
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19028
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19029
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19030
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19031
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19032
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19033
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19034
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19035
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19036
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19037
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19038
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19039
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19040
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19041
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19042
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19043
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19044
2015-11-10 13:19:31 [DEBUG   ] 2015-11-10 13:19:31 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 19045
2015-11-10 13:19:43 [DEBUG   ] 2015-11-10 13:19:43 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19020 exited with status 0
2015-11-10 13:19:43 [DEBUG   ] 2015-11-10 13:19:43 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19021 exited with status 0
2015-11-10 13:19:43 [DEBUG   ] 2015-11-10 13:19:43 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19022 exited with status 0
2015-11-10 13:19:43 [DEBUG   ] 2015-11-10 13:19:43 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19023 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19024 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19025 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19026 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19027 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19028 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19029 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19030 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19031 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19032 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19033 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19034 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19035 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19036 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19037 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19038 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19039 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19040 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19041 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19042 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19043 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19044 exited with status 0
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 19045 exited with status 0
2015-11-10 13:19:44 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] spawn_process: at 2015-11-10 13:19:44 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 19072
2015-11-10 13:19:44 [DEBUG   ] 2015-11-10 13:19:44 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 19073
2015-11-10 13:19:49 [DEBUG   ] 2015-11-10 13:19:49 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 19072 exited with status 0
2015-11-10 13:19:49 [DEBUG   ] 2015-11-10 13:19:49 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 19073 exited with status 0
2015-11-10 13:19:49 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2015-11-10 13:19:49 [DEBUG   ] spawn_process: at 2015-11-10 13:19:49 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2015-11-10 13:19:49 [DEBUG   ] spawn_process: at 2015-11-10 13:19:49 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2015-11-10 13:19:49 [INFO    ] Setting Beam 1 xbars
2015-11-10 13:19:49 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2015-11-10 13:19:49 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2015-11-10 13:19:49 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2015-11-10 13:19:50 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2015-11-10 13:19:50 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2015-11-10 13:19:50 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2015-11-10 13:19:50 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2015-11-10 13:19:50 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2015-11-10 13:19:50 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2015-11-10 13:19:50 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2015-11-10 13:19:50 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2015-11-10 13:19:50 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2015-11-10 13:19:51 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2015-11-10 13:19:51 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2015-11-10 13:19:51 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2015-11-10 13:19:51 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2015-11-10 13:19:51 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2015-11-10 13:19:51 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2015-11-10 13:19:51 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2015-11-10 13:19:51 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2015-11-10 13:19:51 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2015-11-10 13:19:51 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2015-11-10 13:19:52 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2015-11-10 13:19:52 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2015-11-10 13:19:52 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2015-11-10 13:19:52 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2015-11-10 13:19:52 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2015-11-10 13:19:52 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2015-11-10 13:19:52 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2015-11-10 13:19:52 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2015-11-10 13:19:52 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2015-11-10 13:19:53 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2015-11-10 13:19:53 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2015-11-10 13:19:53 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2015-11-10 13:19:53 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2015-11-10 13:19:53 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2015-11-10 13:19:53 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2015-11-10 13:19:53 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2015-11-10 13:19:53 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2015-11-10 13:19:53 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2015-11-10 13:19:54 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2015-11-10 13:19:54 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2015-11-10 13:19:54 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2015-11-10 13:19:54 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2015-11-10 13:19:54 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2015-11-10 13:19:54 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2015-11-10 13:19:54 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2015-11-10 13:19:54 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2015-11-10 13:19:54 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2015-11-10 13:19:55 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2015-11-10 13:19:55 [INFO    ] Setting Beam 2 xbars
2015-11-10 13:19:55 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2015-11-10 13:19:55 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2015-11-10 13:19:55 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2015-11-10 13:19:55 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2015-11-10 13:19:55 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2015-11-10 13:19:55 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2015-11-10 13:19:55 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2015-11-10 13:19:55 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2015-11-10 13:19:56 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2015-11-10 13:19:56 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2015-11-10 13:19:56 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2015-11-10 13:19:56 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2015-11-10 13:19:56 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2015-11-10 13:19:56 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2015-11-10 13:19:56 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2015-11-10 13:19:56 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2015-11-10 13:19:56 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2015-11-10 13:19:57 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2015-11-10 13:19:57 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2015-11-10 13:19:57 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2015-11-10 13:19:57 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2015-11-10 13:19:57 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2015-11-10 13:19:57 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2015-11-10 13:19:57 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2015-11-10 13:19:57 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2015-11-10 13:19:57 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2015-11-10 13:19:58 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2015-11-10 13:19:58 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2015-11-10 13:19:58 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2015-11-10 13:19:58 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2015-11-10 13:19:58 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2015-11-10 13:19:58 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2015-11-10 13:19:58 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2015-11-10 13:19:58 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2015-11-10 13:19:58 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2015-11-10 13:19:58 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2015-11-10 13:19:59 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2015-11-10 13:19:59 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2015-11-10 13:19:59 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2015-11-10 13:19:59 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2015-11-10 13:19:59 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2015-11-10 13:19:59 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2015-11-10 13:19:59 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2015-11-10 13:19:59 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2015-11-10 13:19:59 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2015-11-10 13:20:00 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2015-11-10 13:20:00 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2015-11-10 13:20:00 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2015-11-10 13:20:00 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2015-11-10 13:20:00 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2015-11-10 13:20:00 [INFO    ] Setting Beam 3 xbars
2015-11-10 13:20:00 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2015-11-10 13:20:00 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2015-11-10 13:20:00 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2015-11-10 13:20:00 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2015-11-10 13:20:01 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2015-11-10 13:20:01 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2015-11-10 13:20:01 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2015-11-10 13:20:01 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2015-11-10 13:20:01 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2015-11-10 13:20:01 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2015-11-10 13:20:01 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2015-11-10 13:20:01 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2015-11-10 13:20:01 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2015-11-10 13:20:02 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2015-11-10 13:20:02 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2015-11-10 13:20:02 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2015-11-10 13:20:02 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2015-11-10 13:20:02 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2015-11-10 13:20:02 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2015-11-10 13:20:02 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2015-11-10 13:20:02 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2015-11-10 13:20:02 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2015-11-10 13:20:03 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2015-11-10 13:20:03 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2015-11-10 13:20:03 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2015-11-10 13:20:03 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2015-11-10 13:20:03 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2015-11-10 13:20:03 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2015-11-10 13:20:03 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2015-11-10 13:20:03 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2015-11-10 13:20:03 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2015-11-10 13:20:04 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2015-11-10 13:20:04 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2015-11-10 13:20:04 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2015-11-10 13:20:04 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2015-11-10 13:20:04 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2015-11-10 13:20:04 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2015-11-10 13:20:04 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2015-11-10 13:20:04 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2015-11-10 13:20:04 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2015-11-10 13:20:05 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2015-11-10 13:20:05 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2015-11-10 13:20:05 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2015-11-10 13:20:05 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2015-11-10 13:20:05 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2015-11-10 13:20:05 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2015-11-10 13:20:05 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2015-11-10 13:20:05 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2015-11-10 13:20:05 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2015-11-10 13:20:05 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2015-11-10 13:20:06 [INFO    ] Setting Beam 4 xbars
2015-11-10 13:20:06 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2015-11-10 13:20:06 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2015-11-10 13:20:06 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2015-11-10 13:20:06 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2015-11-10 13:20:06 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2015-11-10 13:20:06 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2015-11-10 13:20:06 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2015-11-10 13:20:06 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2015-11-10 13:20:06 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2015-11-10 13:20:07 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2015-11-10 13:20:07 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2015-11-10 13:20:07 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2015-11-10 13:20:07 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2015-11-10 13:20:07 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2015-11-10 13:20:07 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2015-11-10 13:20:07 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2015-11-10 13:20:07 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2015-11-10 13:20:07 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2015-11-10 13:20:08 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2015-11-10 13:20:08 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2015-11-10 13:20:08 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2015-11-10 13:20:08 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2015-11-10 13:20:08 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2015-11-10 13:20:08 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2015-11-10 13:20:08 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2015-11-10 13:20:08 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2015-11-10 13:20:08 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2015-11-10 13:20:09 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2015-11-10 13:20:09 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2015-11-10 13:20:09 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2015-11-10 13:20:09 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2015-11-10 13:20:09 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2015-11-10 13:20:09 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2015-11-10 13:20:09 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2015-11-10 13:20:09 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2015-11-10 13:20:09 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2015-11-10 13:20:10 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2015-11-10 13:20:10 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2015-11-10 13:20:10 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2015-11-10 13:20:10 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2015-11-10 13:20:10 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2015-11-10 13:20:10 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2015-11-10 13:20:10 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2015-11-10 13:20:10 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2015-11-10 13:20:10 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2015-11-10 13:20:11 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2015-11-10 13:20:11 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2015-11-10 13:20:11 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2015-11-10 13:20:11 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2015-11-10 13:20:11 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2015-11-10 13:20:11 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2015-11-10 13:20:11 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2015-11-10 13:20:11 [INFO    ] All xbars set
2015-11-10 13:20:11 [INFO    ] Setting adder chain routing for all beams
2015-11-10 13:20:16 [INFO    ] Setting beam ids
2015-11-10 13:20:26 [INFO    ] Starting beam delay calibration
2015-11-10 13:20:34 [DEBUG   ] Got command RPT from MCS: ref #536999
2015-11-10 13:20:34 [DEBUG   ] summary = BOOTING
2015-11-10 13:20:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   536999  15 57336 48034215 ABOOTINGBOOTING'
2015-11-10 13:20:39 [INFO    ] Beam 1 latency is 5738
2015-11-10 13:20:39 [INFO    ] Beam 2 latency is 5768
2015-11-10 13:20:39 [INFO    ] Beam 3 latency is 5751
2015-11-10 13:20:39 [INFO    ] Beam 4 latency is 5737
2015-11-10 13:20:39 [INFO    ] Stop calibration
2015-11-10 13:20:46 [INFO    ] Beam delay calibration successful
2015-11-10 13:20:46 [INFO    ] Calibration passed
2015-11-10 13:20:46 [DEBUG   ] spawn_process: at 2015-11-10 13:20:46 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2015-11-10 13:20:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2015-11-10 13:20:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2015-11-10 13:20:47 [DEBUG   ] spawn_process: at 2015-11-10 13:20:47 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2015-11-10 13:20:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2015-11-10 13:20:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2015-11-10 13:20:47 [DEBUG   ] spawn_process: at 2015-11-10 13:20:47 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2015-11-10 13:20:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2015-11-10 13:20:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2015-11-10 13:20:47 [DEBUG   ] spawn_process: at 2015-11-10 13:20:47 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2015-11-10 13:20:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2015-11-10 13:20:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2015-11-10 13:20:47 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -3 msec
2015-11-10 13:20:47 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 350 ms (1447161647.353 s) with PID 20642 and time 1447161648
2015-11-10 13:20:47 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -3 msec
2015-11-10 13:20:47 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 440 ms (1447161647.443 s) with PID 20645 and time 1447161648
2015-11-10 13:20:48 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2015-11-10 13:20:48 [INFO    ] Started antennas statistics background thread
2015-11-10 13:20:48 [INFO    ] Started board flags background thread
2015-11-10 13:20:48 [INFO    ] Started BAM background thread
2015-11-10 13:20:48 [INFO    ] Finished the INI process (ref. #536991) in 108.258 s
2015-11-10 13:20:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 13:20:49 [DEBUG   ] Finished updating board flags in 0.993 seconds
2015-11-10 13:21:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 13:21:28 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 13:21:30 [DEBUG   ] Got command RPT from MCS: ref #537003
2015-11-10 13:21:30 [DEBUG   ] summary = NORMAL
2015-11-10 13:21:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   537003  14 57336 48090303 A NORMALNORMAL'
2015-11-10 13:21:30 [DEBUG   ] Got command RPT from MCS: ref #537004
2015-11-10 13:21:30 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 13:21:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   537004  48 57336 48090303 A NORMALSystem calibrated and operating normally'
2015-11-10 13:21:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 13:21:49 [DEBUG   ] Finished updating board flags in 1.263 seconds
2015-11-10 13:22:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 13:22:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 13:22:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999979
2015-11-10 13:22:49 [DEBUG   ] Got command RPT from MCS: ref #537005
2015-11-10 13:22:49 [DEBUG   ] summary = NORMAL
2015-11-10 13:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   537005  14 57336 48169135 A NORMALNORMAL'
2015-11-10 13:22:49 [DEBUG   ] Got command RPT from MCS: ref #537006
2015-11-10 13:22:49 [DEBUG   ] info = System calibrated and operating normally
2015-11-10 13:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   537006  48 57336 48169135 A NORMALSystem calibrated and operating normally'
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999979
2015-11-10 13:22:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2015-11-10 13:22:49 [DEBUG   ] Finished updating board flags in 1.258 seconds
2015-11-10 13:23:08 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 13:23:28 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 13:23:48 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 1 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 2 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 3 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 4 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa27, chip 5 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 1 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 2 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 3 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 4 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa24, chip 5 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 1 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 2 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 3 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 4 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa23, chip 5 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 1 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 2 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 3 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 4 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa18, chip 5 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 1 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 2 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 3 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 4 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa10, chip 5 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 1 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 2 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 3 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 4 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa32, chip 5 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 1 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 2 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 3 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 4 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa11, chip 5 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 1 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 2 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 3 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 4 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa30, chip 5 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 1 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 2 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 3 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 4 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa12, chip 5 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 1 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 2 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 3 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 4 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa8, chip 5 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 1 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 2 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 3 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 4 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa13, chip 5 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 1 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 2 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 3 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 4 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa26, chip 5 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 1 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 2 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 3 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 4 with 195999999
2015-11-10 13:23:49 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa14, chip 5 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 1 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 2 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 3 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 4 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa22, chip 5 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 1 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 2 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 3 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 4 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa20, chip 5 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 1 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 2 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 3 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 4 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa1, chip 5 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 1 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 2 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 3 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 4 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa5, chip 5 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 1 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 2 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 3 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 4 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa25, chip 5 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 1 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 2 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 3 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 4 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa35, chip 5 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 1 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 2 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 3 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 4 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa19, chip 5 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 1 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 2 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 3 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 4 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa21, chip 5 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 1 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 2 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 3 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 4 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa6, chip 5 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 1 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 2 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 3 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 4 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa7, chip 5 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 1 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 2 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 3 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 4 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa33, chip 5 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 1 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 2 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 3 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 4 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa15, chip 5 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 1 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 2 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 3 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 4 with 195999999
2015-11-10 13:23:50 [WARNING ] MonitorFlags: 1PPS unstable on hostname lwa28, chip 5 with 195999999
2015-11-10 13:23:50 [DEBUG   ] Finished updating board flags in 1.759 seconds
2015-11-10 13:24:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 13:24:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 13:24:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2015-11-10 13:24:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2015-11-10 13:24:50 [DEBUG   ] Finished updating board flags in 1.521 seconds
